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In 1997 the Austin Independent School District (AISD) 

(Texas) launched the Austin Collaborative for Mathematics Education (ACME), a 
project funded by the National Science Foundation to improve mathematics 
education districtwide. The initiative provides all K-8 mathematics teachers 
in the district with long-term professional development and campus-based 
support to implement current national standards for mathematics curriculum 
and instruction. The teaching practices of teachers before and after 
participation in ACME programs were examined. A random sample of 248 
mathematics teachers and 39 special education teachers returned completed 
questionnaires on attitudes toward mathematics reform, teaching practices, 
content knowledge, and demographic characteristics. Ten randomly selected 
classrooms and 3 special education classes were observed. During the ACME 
project's first year, teacher attitudes changed from anxiety to confidence 
and interest in mathematics reform. The investigative culture and design of 
ACME professional development were rated "effective" by teachers. Strong 
points included the professional community of learners ACME supported, and 
the responsiveness of the program to teacher concerns. Some challenges to the 
program were identified, and recommendations are made for program 
improvement. Appendixes contain the teacher questionnaire and study interview 
protocols. (Contains l figure, 5 tables, and 13 references.) (SLD) 
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Overview 

In the summer of 1997, the 
Austin I.S .D. launched the 
Austin Collaborative for 
Mathematics Education 
(ACME), a project funded by 
the National Science 
Foundation (NSF) to improve 
mathematics education 
districtwide. The initiative 
provides all K-8 mathematics 
teachers in the district with 
long-term professional 
development and campus- 
based support to implement 
current national standards for 
mathematics curriculum and 
instruction. Through the 
ACME professional 
development, district 
administrators intend to build a 
community of learners and to 
help teachers explore and 
acquire mathematics content, 
the philosophy of standards- 
based pedagogy, and 
classroom management for 
investigative mathematics 
instruction. In 5 years, over 
2000 teachers, including 
general and special education, 
bilingual education, and 
English as a Second Language 
(ESL) teachers, will participate 
in the ACME project. 

The purpose of this evaluation 
is to examine the teaching 
practices of educators before 
they participate in the ACME 
professional development and 
to describe district teachers’ 



and principals’ attitudes 
toward mathematics reforms, 
the quality of ACME 
professional development, and 
the supports in the district and 
community to implement the 
project in its first year. 

Methodology 

Of a random sample of K-8 
teachers, 248 mathematics and 
39 special education teachers 
returned completed 
questionnaires on attitudes 
toward mathematics reforms, 
teaching practices, district 
support, mathematics content 
knowledge, and demographic 
characteristics. All 82 
principals of the district’s 
elementary and middle schools 
completed questionnaires about 
school reforms, attitudes 
towards standards-based 
mathematics and science 
education, support for the 
ACME project, and school 
demographics. Ten randomly 
selected mathematics 
classrooms and 3 special 
education classrooms were 
observed and rated for the 
effectiveness of the instruction. 
Eight professional development 
sessions were observed and 
rated for quality and 
effectiveness. In addition, 
district and ACME project 
staff were interviewed about 
their experiences with the 
project and opinions about its 
progress. 



Major Findings 

Strengths of ACME 

• Most teachers new to 
ACME and principals 
were familiar with the 
language of standards- 
based mathematics and 
highly endorsed its 
practices. 

• During the ACME 
project’s first year, the 
attitudes of teachers who 
participated in ACME 
professional development 
transformed from anxiety 
and apprehension to 
confidence and interest. 

• The investigative culture 
and design of ACME 
professional development 
were primarily rated 
effective for supporting 
teachers’ efforts to 
enhance children’s 
mathematical competence. 

• Strong points include the 
professional community of 
learners that the ACME 
project has established, the 
collaboration it encouraged 
within and across 
campuses, and its 
responsiveness to teachers’ 
concerns. 

• Additional strong points 
include the alignment of 
district policies and 
curriculum as well as 
financial and material 
backing for the project. 



Challenges of ACME 

• Determine where deep 
understanding of 
standards-based 
mathematics education is 
lacking in the face of 
familiarity with the 
language of current 
standards for mathematics 
education in the district; 
teachers’ descriptions of 
their teaching practices 
and observer ratings may 
differ 

• Raise the quality of 
implementation, 
mathematics content, and 
pedagogy in ACME 
professional development, 
especially as the project 
adds teacher leaders who 
have little experience as 
facilitators. 

• Maintain full teacher 
involvement over the 
course of ACME 
professional development 
and gamer the full 
involvement and leadership 
of school administrators. 

• School administrators and 
teachers need to agree on 
time for teachers to 
collaborate about 
mathematics education as 
well as promote more 
parent involvement in 
mathematics education. 

• A particular challenge to 
ACME is the high stakes 
state and district 
assessment system. 

• Establish support 
structures that will sustain 
high quality professional 
development in the district 
after ACME is complete 



Recommendation 

• Foster among the district’s 
teachers, principals, 
central office 
administrators, and the 
community an 
understanding of the 
difference between 
superficial implementation 
and standards-based 
instruction that enhances 
children’s mathematical 
competence by building 
bridges between 
experienced and 
inexperienced educators. 

• Continue to address 
teachers’ concerns about 
implementation through 
the framework of the 
Concems-Based Adoption 
Model. 

• Continue to provide 
teachers and principals 
with ongoing high quality 
professional development 
and to promote the 
professional community of 
learners that is emerging 

• Campaign for full 
participation of teachers 
and administrators in 
ACME professional 
development. 

• Teachers need to make use 
of time set aside for 
collaboration on 
mathematics education and 
share innovative strategies 
that promote collaboration 
across campuses. 

• Continue to address 
concerns about the 
relationship between 
standards-based 



mathematics curriculum 
and instruction and the 
district and state 
assessment system. 

• Share lessons learned from 
the evolution of the ACME 
professional development 
system with the district’s 
department of professional 
development and develop a 
powerful, self-sustaining 
program 



For a copy of the full 
report 97.18, contact: 
Austin I.S.D. 

Office of Program 
Evaluation 

1111 West Sixth Street 
Austin, Texas 
78703-5399 
(512) 414-1724 
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Overview of the Project 

In August of 1997, the Austin Independent School District (AISD) launched a 
Local Systemic Change initiative to implement national standards for mathematics 
curriculum and instruction in all K-8 classrooms. This district initiative, the Austin 
Collaboration for Mathematics Education (ACME), is funded in part by the National 
Science Foundation (NSF), and is a partnership with the Charles A. Dana Center and the 
University of Texas at Austin. ACME is a unique approach in that it provides systematic 
and continuing professional development for every elementary and middle school 
mathematics teachers in a large urban district. 

Through ACME, each K-8 mathematics teacher in the district participates in a 
minimum of 120 hours of formal professional development including summer institutes, 
follow-up training during the academic year, and campus level support such as peer 
coaching, collegial sharing, and modeling. The intent of professional development is (1) 
to build sound practice in mathematics instruction, (2) to ensure consistent implementation 
of a quality mathematics curriculum, and (3) to provide ongoing support for teachers arid 
administrators to implement standards-based curriculum and instruction in every 
classroom in the district. 

The initiative takes place in a growing urban district with over 75,000 students, 
approximately 55,000 of whom are of elementary and middle school age. The project 
currently works directly with 68 elementary schools and 1 5 middle schools. The size of 
schools is diverse, ranging from 500 to 1700 students for middle schools and 275 to 1300 
students for elementary schools. The district serves a racially and ethnically diverse 
student population. Forty-three percent of the students are Hispanic, 37% are Anglo, 
18% are African American, 2% are Asian or Pacific Islander and less than 1% are Native 
American. Among the students, 14% have limited English proficiency and 50% qualify for 
free or reduced lunches through federal funds. 

The major goal of the project is to raise the mathematics achievement of all 
students in the district by implementing a rigorous, standards-based curriculum for every 
child. The Texas State Accountability System has revealed inequities in students’ 
mathematics achievement through their performance on the Texas Assessment of 
Academic Skills (TAAS). Although in recent years the percentage of students who pass 
the mathematics segment of the TAAS test has increased, students who are African 
American, Hispanic, or economically disadvantaged consistently perform below the 
average for all students (Williams, 1998). District administrators recognize the need to 
provide students with the mathematics understanding, reasoning, and communication skills 
that move beyond mathematics computation. 

To address these needs, the initiative assists elementary and middle school staff in 
implementing the district curriculum that is based on standards designated by the National 
Council of Teachers of Mathematics (NCTM, 1989, 1991, 1995.) District staff are 
purchasing the curriculum resources of Investigations in Number, Data, and Space for 
elementary grades and Connected Mathematics for middle grades to support teachers’ 
implementation of standards-based instruction. These curriculum resources are 
particularly well suited for AISD because they support the following teaching practices 
(cf. Russell, 1998): 
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• Promoting children’s mathematical thinking, reasoning, and problem-solving 
skills; 

• Developing children’s deep understanding of mathematical concepts through 
hands-on experiences, real-world problems, and communication; and 

• Supporting a vertically and horizontally coordinated curriculum that considers 
the needs of all students, including those who are special education, gifted and 
talented, limited English proficient, and bilingual. 

These investigative practices emphasize mathematical literacy through the understanding 
of mathematics concepts and approach instruction through problem-solving and 
communication of mathematical ideas. These practices contrast with traditional practices 
that emphasize mathematical algorithms, rote memorization, and mastery of computation 
(Cohen & Ball, 1990). 

To support improvement in students’ mathematical competence in every 
classroom, district administrators also recognize the need to build a community of learners 
that fosters teachers’ professional growth as mathematics instructors. The ACME project 
provides long-term professional development designed to facilitate teachers’ exploration 
and acquisition of mathematics content, the philosophy behind standards-based pedagogy, 
and classroom management for investigative, inquiry-based mathematics instruction. 

Through the ACME project, each of the district’s K-8 mathematics teacher, 
participates in a series of professional development activities over three years. The more 
than 2000 participants include general education, special education, bilingual, and English 
as a Second Language (ESL) teachers. The first two years include week-long summer 
institutes and five follow-up days during each academic year. The third year involves less 
formal professional development to continue through campus initiatives that the project 
will support. Teachers are paid a stipend to attend the summer institutes and follow-up 
sessions outside school hours, and substitutes are provided to release teachers during the 
academic year. 

The implementation plan begins at the transition between elementary and middle 
school. Thus, in the summer of 1997, staff development began with fifth and sixth grade 
teachers, followed by fourth and seventh grade teachers in the summer of 1998, and will 
continue with kindergarten through third grade and eighth grade teachers in subsequent 
years. In addition to grade-by-grade implementation, eight elementary schools and three 
middle schools are serving as pilot sites with all grade levels participating in the initiative 
simultaneously beginning in year one of the project. 

A second goal of the initiative is to establish systemic change by developing school 
cultures in which communities continually improve mathematics teaching and learning. In 
the summer of 1998, ACME provided an institute for principals and their assistants that 
addressed administrative issues such as strategies for supporting teachers in 
implementation of standards based curriculum and instruction. The project also makes use 
of organizational structures already present (e.g., curriculum specialists) to support 
teachers on campuses. Through these campus leaders, the project disseminates 
information to teachers on every campus. The ACME project staff is also developing 
customized professional development for lead teachers so that they may facilitate 
professional development sessions and support their peers on the campus level in a variety 
of ways, including peer coaching, demonstration teaching, information sources, and 
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sharing classroom management techniques. To garner parent participation in the 
mathematics curriculum, the project staff provides schools with technical support (e g., 
pamphlets and videos in English and Spanish) as well as assistance with organizing parent 
education and involvement (e g., parent math nights). 

Additionally, the project staff enlists support from the district’s administrative 
leaders. Central and campus administrators have undergone numerous changes over the 
past five years, including the resignation of the superintendent and appointment of an 
interim superintendent in April of 1998. Throughout these changes, support for the 
ACME project has continued. 

A third and final goal of the project is to ensure that all of the district’s resources 
and efforts in mathematics education are aligned with local, state, and national standards. 
ACME project staff assert that the district’s mathematics curriculum document, the state 
standards in the Texas Essential Knowledge and Skills (TEKS), and the NCTM (1989, 
1991, 1995) standards are aligned. To ensure that these standards emerge in the district’s 
mathematics classrooms, the ACME project channels local, state, and federal funds to 
teachers’ professional development and to support the district’s textbook adoption of the 
curriculum resources. 
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Methodology 



Purpose of Evaluation 

NSF requires an evaluation of ACME in collaboration with Horizon Research, Inc. 
(HRI), an educational research group contracted to design the national evaluation of 
mathematics and science systemic change initiatives. This evaluation is formative, and 
provides information about project implementation so that decision makers can make 
improvements. The purpose of this report is to establish a baseline for what mathematics 
education and instruction looks like in the district and the current supports for 
implementation. The report presents attitudes of teachers and principals toward 
mathematics reforms, documents mathematics classroom instruction, examines the quality 
of teachers’ professional development, and assesses the extent of support for the project 
districtwide. 

Sample and Procedures 

To establish a clear picture of mathematics education and instruction throughout 
the district in the ACME project’s baseline year, data were gathered from multiple sources 
and in various formats that are described next. 

Teacher Questionnaires 

The local evaluator prepared a list of all elementary and middle school mathematics 
teachers who were employed by the district as of February of 1998. A previous survey 
revealed that some of the district’s elementary schools departmentalize grade levels such 
that some educators teach language arts and/or social studies but not mathematics 
(Batchelder, 1998). Therefore, the evaluator phoned campus administrators to screen the 
list for mathematics teachers only. From this list, HRI selected a random sample of 300 
mathematics teachers, and the evaluator mailed questionnaires to these teachers. These 
questionnaires, designed by HRI, survey teachers’ attitudes toward national standards for 
mathematics instruction, preparation to implement the standards, classroom practice, 
mathematics content knowledge, perceptions of district support, demographic 
characteristics, and experiences in ACME professional development. (See Appendices for 
a sample teacher questionnaire.) 

A few weeks after the first mailing of questionnaires in April of 1998, 50% of the 
teachers had returned the forms. A second mailing was sent out in May to teachers who 
had not yet returned a completed questionnaire. Additionally, teachers who had returned 
the questionnaire but skipped some items were sent photocopies of the pages with missing 
information and a request to return the completed copies. At the beginning of May, 
principals were given a list of teachers at their school who had not yet returned the forms 
and were told to ensure that the forms were returned. Out of the original 300 
mathematics teachers in the sample, 1 1 were omitted for various reasons such as not 
teaching mathematics, parenting or maternity leave, resignation, and retirement. A total of 
248 mathematics teachers out of the 289 valid respondents returned completed 
questionnaires yielding a response rate of 86%. 
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In addition to these mathematics teachers, a sample of 50 special education 
teachers was sent questionnaires. Four teachers were omitted because they did not teach 
mathematics, had resigned, or were not presently teaching special education. Thirty-nine 
special education teachers out of the 46 valid respondents returned questionnaires with a 
response rate of 85%. 

Additional sources of information from teachers included informal interviews and 
observations during professional development. The ACME project staff also designed 
questionnaires for teachers to evaluate the professional development. Teachers’ responses 
to these evaluations provided information about enhancement of their knowledge of 
mathematics content, strengths of the sessions, concerns about implementing the 
standards-based curriculum and instruction, and suggestions for improving the project. 

Principal Questionnaires 

In April of 1998, a survey was sent to the principals of all of the 15 middle schools 
and the 66 elementary schools, as well as to the principal of the Alternative Learning 
Center. The principal questionnaires assessed the extent of school reforms, attitudes 
toward standards-based curricula in mathematics and science, district and state support for 
the ACME project, familiarity with the project, and school demographics. (See 
Appendices for a sample principal questionnaire.) Data collection procedures for the 
principals who did not complete their questionnaires paralleled those used for teachers. In 
mid-May, the evaluator called principals who had not yet returned their forms. All 82 
principals returned the questionnaires for a 100% response rate. 

Classroom Observations 

From the sample of 300 elementary and middle school mathematics teachers, HRI 
randomly selected 10 teachers, none of whom had participated in ACME professional 
development, for classroom observation. One back-up teacher was observed because one 
teacher in the original sample had been pulled from the classroom to teach a literacy 
project full-time. All of the observed classrooms were kindergarten through fourth grade 
classes, except one, which was a seventh grade class. The lead evaluator also drew a 
random sample of four special education teachers to observe. One of these teachers was 
on leave and another declined to be observed because she was going on maternity leave. 
A back-up special education teacher was randomly selected so that a total of three special 
education classrooms could be observed. For all of the observations except two in which 
there were scheduling conflicts, an entire mathematics lesson was observed. 

In March of 1998, the lead evaluator and ACME project manager were both 
trained and certified to rate classroom observations reliably using HRI’s classroom 
observation protocol (HRI, 1998; see Appendices for a sample protocol and pre- and 
post-classroom observation interviews). The lead evaluator rated all of the classroom 
observations except one, which the project manager rated. The protocol is used to rate a 
classroom observation on an ordinal scale, which consists of five global categories that are 
qualitatively different and describe the extent to which a teacher’s mathematics instruction 
is effective and meets national standards. 

The purpose of the classroom observations in the baseline year is to capture a 
snapshot of mathematics instruction in the district before teachers participate in ACME 
professional development. It is important to note that the small sample of 10 observations 
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is not representative of all mathematics instruction in the district; the frequencies and 
proportions of the categories do not generalize to the district as a whole. Rather, these 
observations generally depict some of the different types of instruction in the district as the 
project launches, not the relative frequency of the types. 

Professional Development Observations 

The lead evaluator observed 16 professional development sessions to assess quality 
and content and formally rated 8 of the sessions with HRI’s professional development 
observation protocol. (See Appendices for a sample protocol.) This protocol is similar to 
the one for classroom observation protocol in that it results in global ratings on a 5-point 
ordinal scale ranging from ineffective to exemplary facilitation but for adult learners. The 
eight sessions observed represented three different formats of the project’s professional 
development. One session was for school administrators and provided logistical and 
conceptual information about the new mathematics curriculum and instruction, 
suggestions for answering parent concerns, support for teachers, and so on. Another 
session was a follow-up during the academic year for fifth and sixth grade teachers to 
compare their materials. The other six sessions observed were week-long summer 
institutes that covered pedagogical and mathematics content for fourth grade teachers in 
their first year of the ACME project and fifth grade teachers, most of whom were in their 
second year. The evaluator conducted these additional sessions to meet the project 
director’s request for a complete assessment of professional development provided by all 
of the facilitators. For the remaining sessions, the evaluator was a participant observer. 
Two of these observations included the summer institutes for sixth and seventh grade 
teachers for which the project hired facilitators from Michigan who have extensive 
experience as trainers and have test-piloted CMP in their classrooms. Observations that 
were rated lasted from one hour to one-half of a day, and informal observations lasted 
from one hour to one-half of a day or longer. 

Interviews with District and Project Staff 

This evaluation also included information about project procedures and its history 
as well as district policy gleaned from semi-structured interviews with district 
administrators and project staff. These sources include the deputy superintendent of 
instructional services and school operations, the mathematics supervisor and ACME 
project director, the project manager, and a mathematics specialist who facilitates 
professional development for all district employees. A project director of the State 
Systemic Initiative (SSI) at the Dana Center reported on the project’s history and design. 
This person has been instrumental in planning the project and participated in an early 
mathematics reform effort in the district as an area superintendent. In addition, the lead 
evaluator participated in the ACME project meetings, in a retreat at which the facilitators 
reflected on the first year of implementation, and in informal conversations with project 
staff. These interviews and observations provided different perspectives of the project’s 
functioning in the baseline year. 

Additional sources of data for this report include the district’s grant proposal to 
NSF and subsequent revisions, the district and state mathematics curriculum documents, 
professional development materials, and brochures for parents. 
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Results and Discussion 

The findings presented in this section integrate different perspectives on the 
ACME project, including the viewpoints of teachers, principals, and central office staff 
This section includes data on the district’s mathematics curriculum and instruction in the 
ACME project’s baseline year, the quality of professional development provided, and the 
supports for sustaining the project. 

In general, the data from teacher questionnaires were taken from the responses of 
the 289 mathematics teachers because in many cases their responses were not different 
from those of the 39 special education teachers who completed questionnaires. Where the 
differences in the responses of these two teacher samples were statistically significant, the 
discrepancies were noted. 

Teachers’ Attitudes Toward and Preparation for Mathematics Reforms 

Figure 1. A Comparison of the Percentage of Teachers Who Responded 
Very Important and Very Well Prepared for Use of Specific Instructional Strategies 




Instructional Strategies 



Source: Teacher questionnaires 

Attitudes Toward and Preparation in Instructional Strategies 

Many of the teachers who responded to the questionnaire highly endorsed 
investigative strategies in mathematics instruction. They rated a variety of these strategies 
as very important for effective mathematics instruction (see Figure 1). Over 80% of the 
teachers placed high importance on hands-on activities, providing concrete experiences 
before abstract concepts, making connections between mathematics and other disciplines, 
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and applying mathematics in a variety of contexts. More than 60% of these teachers 
reported that they found cooperative learning and inquiry-oriented activities very 
important to effective mathematics instruction. Over half of the teachers valued highly the 
use of computers in mathematics instruction. 

Although many teachers endorsed instructional strategies that characterize 
standards-based mathematics, fewer teachers felt highly prepared to use these strategies 
(see Figure 1). Less than half of the teachers reported feeling very well prepared to direct 
students in hands-on activities and cooperative learning, for example. Moreover, less than 
one-third of these teachers reportedly felt very well prepared to facilitate inquiry-oriented 
activities or to use computers in their mathematics lessons. Thus, according to teachers’ 
reports of their teaching preparation, many lack preparation to teach children successfully 
through the instructional strategies of standards-based mathematics. 

Preparation in Mathematics Content 

In general, many of the mathematics teachers reported feeling well prepared to 
teach mathematics, including the mathematics content traditionally taught in U S. 
elementary and middle schools. Many of these teachers (58%) reported feeling very well 
prepared to teach mathematics compared with few (27%) who felt very well prepared to 
teach science. As a majority of these teachers (82%) taught in elementary schools, the 
mathematics content that they felt comfortable with reflects their work experience (see 
Table 1). More than half of the mathematics teachers reportedly felt very well prepared to 
teach computation, patterns and relationships, measurement, numeration and number 
theory, and estimation. Small proportions of these teachers felt very well prepared to 
teach subjects such as probability and pre-algebra, which broaden the mathematical 
knowledge of elementary students in accordance with current standards. Thus, a minority 
of the mathematics teachers who responded felt well prepared to teach mathematics 
content that traditionally has received little attention. 

Fewer special education teachers than mathematics teachers reported feeling well 
prepared to teach mathematics and mathematics content that U.S. elementary and middle 
schools have traditionally overlooked. One-quarter (26%) of the special education 
teachers reported feeling very well prepared to teach mathematics, whereas less than one- 
tenth (8%) of these teachers felt very well prepared to teach science. Although more than 
half (56%) of the special education teachers felt very well prepared to teach traditional 
mathematics content, such as computation, approximately one-third of these teachers felt 
very well prepared to teach patterns and relationships, measurement, numeration and 
number theory, and estimation. Similarly, less than one-fourth of the special education 
teachers felt very well prepared to teach topics traditionally overlooked such as geometry 
and data collection, and one-tenth of these teachers felt very well prepared to teach 
probability and use calculators and computers in mathematics classes. In sum, a small 
majority of these special education teachers feel comfortable teaching computation and 
some felt comfortable with some traditional content. For the most part, these special 
education teachers do not report feeling prepared to teach any other areas of mathematics, 
especially areas that expand the curriculum beyond traditional content areas. 
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Table 1. Percentage of K-8 Mathematics Teachers Prepared in Mathematics Topics 
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Source : Teacher questionnaires 



Relationship Between Teachers’ Attitudes and Preparation 

Teachers’ endorsement of standard s-based instructional strategies could be 
interpreted in different ways. One hypothesis is that many teachers who have not yet 
participated in training highly value investigative instructional strategies and would like to 
use them in their classrooms. Perhaps, these teachers will welcome the new curriculum 
and instruction. On the other hand, the findings that many teachers are not well prepared 
to carry out these activities imply that many will require some professional development to 
be able to implement them in their classrooms. Teachers’ endorsement of these 
instructional strategies in combination with their lack of preparation may suggest that 
many are eager to learn these types of instructional strategies. 

Supporting evidence for the eagerness to learn how to implement standard s-based 
instructional strategies comes from informal conversations with teachers and school 
administrators. When observing teachers’ classrooms, the evaluator learned that 
kindergarten through fourth grade teachers who were scheduled to participate in ACME 
professional development in subsequent years of the project were anxiously awaiting the 
opportunity. One fourth grade teacher said that she had seen what a trained fifth grade 
teacher had been doing in her mathematics classroom and looked forward to learning more 
about the new curriculum and instruction herself. A first grade teacher was disappointed 
to find out that she would have to wait two years before participating in ACME 
professional development. 

Similarly, some schools are not waiting for the planned grade-by-grade 
implementation to reach all of their teachers. Although the district is committed to buying 
curriculum materials for all campuses before each grade level begins professional 
development, ACME program staff reported that some schools have bought the materials 
with campus funds for all grade levels in advance. Because on some campuses all of the 
teachers wanted to participate, not just the fifth and sixth grade teachers who were 
scheduled, schools requested professional development for teachers in other grade levels. 
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ACME program staff decided that they could not include extra participants, and some 
school staffs were disappointed. On at least one campus, the administrator hired outside 
staff developers to teach their teachers, who were not yet targeted for ACME professional 
development, how to implement investigative mathematics curriculum and instruction. 

Thus, it appears that some teachers’ eagerness to learn how to implement the 
standards-based curriculum and instruction may explain, in part, the high level of 
endorsement of these strategies coupled with their reported lack of preparation to use 
these strategies. Moreover, the eagerness of some teachers who must wait for 
professional development suggests that a momentum may build as some motivated 
teachers anxiously anticipate their turn to participate and to try out standards-based 
curriculum and instruction. 

Another hypothesis is that teachers’ endorsement of the standards-based 
curriculum may reflect familiarity with the language of new trends in mathematics 
education, but teachers may lack a clear understanding of the implications of these 
strategies for classroom activities and children’s learning. These data were gathered 
through teachers’ responses to questionnaire items. Many teachers may be familiar with 
phrases like “hands-on activities” or “cooperative learning” and readily rate these items as 
very important. Cohen ( 1 990) found that although teachers may be able to communicate 
in the language of mathematics reforms, their classroom instruction may be dominated by 
traditional strategies. Thus, the findings that teachers highly endorse standards-based 
strategies may be misleading because the data are self-report. Teachers’ endorsement 
does not necessarily imply that these strategies will be evident in their teaching practice. 

Other evidence from teachers’ questionnaires points to their lack of information 
about mathematics reforms. Before participating in the ACME project, some teachers in 
the district were not familiar with NCTM standards. Twenty-two percent of the 
mathematics teachers and 29% of the special education teachers did not consider 
themselves well-informed about the NCTM Standards for the grades they teach, and 21% 
and 32%, respectively, had no opinion. Consequently, 43% of the mathematics teachers 
and 61% of the special education teachers who responded did not assert that they knew 
the standards well. Although some teachers who have not participated in the project 
endorse the language of standards-based instructional strategies, they may not know the 
philosophy behind the strategies nor how to use the techniques effectively to help children 
reason mathematically. This finding supports the hypothesis that teachers may be familiar 
with the language of standards-based strategies, but not understand the implications of 
these strategies for teaching and learning. 

Project staff discerned this lack of knowledge about standards-based mathematics 
education during the first year of the project. One staff member said, 

We had to examine where our teachers are. Our teachers didn ’t 
know NCTM standards or understand problem-solving. When we 
launched the professional development, we assumed that teachers 
knew why “drill and kill ” was not O.K. We had to go back and 
talk about it, go back to the foundations. 

Source: Project staff interviews 
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Through interaction with teachers in ACME professional development, the mathematics 
team learned the limits to teachers’ knowledge about the NCTM standards and its 
philosophy and decided to address the lack of knowledge. However, this lack of 
knowledge supports the supposition that some untrained teachers do not have a 
comprehensive understanding of the standards-based teaching practice. Again, teachers’ 
endorsement of standards-based instructional strategies may reflect familiarity with the 
terms rather than a deep understanding of the implications for teaching practices and 
children’s learning. 

Similar to some teachers’ lack of knowledge of NCTM standards, during, the first 
year of the project, the team recognized limits to teachers’ mathematics content 
knowledge. In teachers’ evaluations of how their mathematics content was enhanced by 
the summer institute of 1998, some acknowledged that they made improvements in this 
strand over the course of the institute. For example, teachers stated that they gained a 
“better understanding of math,” became “more confident with my own understanding of 
math,” and “learned different ways to approach problems.” On the other hand, other 
teachers stated that they did not gain content knowledge, but described instructional 
strategies that they gained instead. Thus, in evaluating what teachers learned in 
professional development, some acknowledged gaining a deeper understanding of 
mathematics than they had before ACME professional development, but other teachers 
seemed to emphasize developing new strategies and approaches to solve mathematics 
problems. 

Interviews with the ACME project staff and observations of teachers at 
professional development suggest that teachers’ attitudes toward standards-based 
instructional strategies are more complex than the responses to questionnaires imply. 
When the project began in the summer of 1997, one staff member described teachers’ 
attitudes as anxious, apprehensive, and cautious. These attitudes are supported by the 
observations of hesitance in some fourth grade teachers at the beginning of their first 
summer institute. Moreover, teachers’ attitudes ranged from opposition to the district’s 
approach to curriculum and instruction to interest and desire for support. Thus, behind 
teachers’ endorsement of standards-based instructional strategies, there may be a mixture 
of anxiety and interest. 

Over the course of the first year, the same staff member who noted a range of 
attitudes also observed a great deal of change in teachers’ attitudes toward the new 
curriculum. At the end of the first year of the project and the second summer institute, 
fifth grade teachers who attended training showed more confidence and interest in the 
curriculum and instructional strategies. Additionally, a higher proportion of the teachers 
were enthusiastic about the project than were the previous summer. Consequently, 
through professional development, facilitators may have helped teachers become more 
prepared to teach standards-based mathematics by supporting and addressing their 
concerns as they tried out the new materials and strategies in their classrooms. 

Classroom Instruction in the Baseline Year 

To assess the characteristics of classroom instruction in the district, the evaluation 
included teachers’ reports on their practice taken from the questionnaires as well as the 
evaluator’s ratings of classroom observations. In general, most teachers report that their 
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teaching practices include a combination of investigative and traditional instructional 
strategies. However, classroom observations reveal that mathematics instruction in the 
baseline year of the project is generally characterized by instruction that contains elements 
of standards-based practice but these are primarily ineffective. Many teachers do not yet 
have the skills necessary to implement the standards. Moreover, a number of teachers 
emphasize practicing computation for mastery and rote memorization. Despite these 
general trends, there is evidence that the teaching practices of some teachers include 
effective standards-based instruction. 

Teachers’ Reports on Mathematics Instruction 

Overall, teachers who responded to the questionnaires emphasized teaching of 
mathematics in their classrooms. 1 Over 75% of the teachers who taught in self-contained 
classrooms reported that they teach five mathematics lessons per week. They also 
reported that the mathematics lessons typically lasted between 41 and 60 minutes for 
approximately half of the classrooms, 61 minutes or more for about one-third of the 
classrooms, but 40 minutes or less for about one-sixth of the classrooms. Compared with 
other subjects, teachers reported that they covered mathematics nearly as often as 
reading/language arts. When reporting what happened in the last five school days in their 
classrooms, over 80% of the teachers taught mathematics, and over 90% taught 
reading/language arts. In contrast, science was taught in the last five schools days by 39% 
of the mathematics teachers who responded, and social studies was taught in the last five 
school days by only 28% of these teachers. No special education teacher with a self- 
contained classroom taught science or social studies in the last five school days. This 
evidence suggests that in the baseline year of the project most students in the district are 
receiving mathematics and language arts instruction almost daily and mathematics lasts a 
considerable amount of time. Science and social studies are covered much less and not at 
all for students with disabilities. This emphasis on mathematics as well as on 
reading/language arts reflects district policy to improve children’s performance in 
mathematics, reading, and writing, the three subject areas tested by the state’s 
accountability measure, the TAAS. 

Teachers’ reports of the characteristics of their instructional strategies echo their 
endorsement of standards-based practices (see Table 2), although special education 
teachers reported slightly less investigative practice than general education teachers. 
However, all teachers’ reports of children’s classroom activities suggest that the 
mathematics lessons include traditional practices as well. Most teachers surveyed reported 
that an investigative culture characterizes their mathematics instruction. For 80% or more 
of the teachers, their mathematics instruction often or always encouraged children to 
communicate mathematically, explain their reasoning, explore alternative solutions, and 
discuss their work. (The responses of special education teachers differed slightly in that a 
small majority, 74%, of these teachers reported encouraging students to communicate 
mathematically.) On the other hand, many teachers also reported practices that 



1 This analysis includes only the teachers who taught in self-contained classrooms (188 of the 248 
mathematics teachers and 12 of the 39 special education teachers who returned the questionnaires). Some 
elementary schools in the district departmentalize grade levels such that teachers teach only one or two 
subjects. 
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characterize traditional teaching practices. For example, more than 70% of these teachers 
indicated that their mathematics instruction often or always included children’s practicing 
algorithms and routine computation as well as reviewing homework and worksheets in 
class. One traditional strategy that these teachers used infrequently was short answer 
tests; only 19% of the mathematics teachers but 31% of the special education teachers 
reported assessing children’s competence with these tests often or always. Thus, before 
participating in ACME professional development, many of these teachers reportedly were 
implementing a combination of traditional and standards-based instructional strategies in 
their mathematics lessons. 

Table 2. Comparison of Teacher-Reported Teacher and Child Behavior in 
Investigative and Traditional Practice: Percentage Reporting in Each Category 
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The finding that many teachers reported implementing traditional as well as 
standards-based strategies could be interpreted in at least two ways. On the one hand, 
perhaps many teachers divide their mathematics lessons into segments such that one part 
involves investigative practice and another part involves traditional practice. In the 
classroom observations, a few teachers indeed divided their mathematics lessons into 
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segments with both investigative and traditional instructional strategies. For example, one 
teacher demonstrated how to represent double-digit addition with ones and tens blocks, 
had her students play an addition game with those materials in small groups, and then 
passed out worksheets for them to practice computation. On the other hand, it is possible 
that teachers describe their instructional strategies in the terms of standards-based practice 
(e g., encouraging children to communicate mathematically or to explain their reasoning) 
without understanding how those strategies can be carried out effectively to influence 
children’s mathematics competence. 

Further examination of teachers’ reports of the activities that children do in their 
mathematics classes revealed a mixture of strategies (see Table 2). Most of the teachers 
indicated that children engage in hands-on mathematics activities often or always, and 
many teachers reported that children solve real-world problems with mathematics often or 
always in their classrooms. However, some specific classroom activities that distinguish 
standards-based mathematics from traditional strategies occur less frequently. For 
example, in less than one-fifth of the classrooms, students often or always worked on an 
investigation or project that lasted a week or more. In approximately one-fourth or less of 
the classrooms, students often or always worked on portfolios, models or simulations, or 
designed or implemented their own investigations. Similarly, in approximately one-fourth 
of the mathematics classrooms and in one-seventh of the special education classrooms did 
standards-based assessments in which students provide descriptions or justifications of 
solutions occur often or always. In contrast, the frequency of children’s work with 
computers and calculators that teachers reported varied. 

Thus, teachers described their own teaching as a combination of standards-based 
mathematics instruction and traditional practice. However, the findings are inconsistent 
and probably do not reflect true standards-based instruction. Teachers reported that 
children were infrequently engaged in activities that characterize investigative mathematics 
in their classrooms and that children often practice computation and review worksheet 
assignments. It is possible that teachers employ strategies associated with investigative 
practice while children engage in traditional activities. For example, a child could be 
completing addition problems and get help from the teacher. During the interaction, the 
teacher may encourage the child to explain her reasoning and to communicate 
mathematically. In this way, teachers could report their own behavior and children’s 
activities in terms of both standards-based and traditional practice. In light of these 
inconsistencies, it is unclear from teachers’ self-reports the degree to which standards- 
based instruction occurs in the district. The following discussion of direct observations 
from a small random sample of mathematics classrooms in the district illustrates a 
discrepancy in teachers’ self-reports of their teaching practices and what actually takes 
place in the district’s mathematics classes. 

Classroom Observations 

Ten observations of mathematics classrooms and three observations of special 
education classrooms provide a snapshot of what teaching practices look like in the 
ACME project’s baseline year. 

Most of the lessons observed in general education classrooms (7 out of 10) 
covered the basic mathematics content of computation and numeration and number 
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theory. However, some of the lessons (4 out of 10) covered topics that traditionally 
receive little attention, such as probability and data collection and analysis. The intended 
purpose of many of the lessons (7 out of 10) was to involve children in higher level 
thinking through developing and reviewing mathematical concepts. However, some of the 
observations (4 out of 10) involved traditional lesson designs that focused on children’s 
memorizing facts, practicing computational algorithms, and drilling addition and 
subtraction for mastery. Therefore, the mathematics content of the observed lessons were 
more often traditional than standards-based, but the intended purpose of the lessons was 
more often standards-based than traditional. 

Teachers’ groupings of the children for most of the class activities varied. In the 
observed lessons of mathematics classrooms, teachers usually organized the children as 
whole groups (8 out of 10) or as individuals (4 out of 10). Several teachers also had 
children work together in small groups or pairs (4 out of 1 0). 

The instructional activities that teachers chose for the observed lessons suggest 
that before implementation of the project teachers used a combination of standards-based 
and traditional approaches. Table 3 presents frequencies of instructional activities that 
were observed in each classroom for a large portion of the lesson. In most of the 
observed lessons (8 out of 10), teachers led class discussions and involved children in 
exploration of mathematical topics. In about one-third of the observations, children were 
passive participants while teachers presented information. For more than half of the 
observations, children were involved in activities that had investigative qualities. For 
example, they worked with manipulatives, recorded and analyzed data, or played games to 
develop knowledge or skills. Traditional practices, such as children’s answering textbook 
or workbook problems, occurred in fewer of the classroom observations. 

Table 3. Frequencies of Instructional Activities in Observed Classrooms 
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Source: Classroom observation protocols 

Despite the finding that many of these teachers organized their mathematics 
lessons around hands-on activities, the results of the classroom observations indicate that a 
majority of teachers were not implementing effective, standards-based mathematics. 
Although teachers may include manipulatives, the quality of the instruction determines 
whether the lesson effectively helps children develop mathematical competence. 
Classroom observations were rated on HRI’s (1998) 5-point scale ranging from effective 
to ineffective instruction. Level 1 refers to ineffective instruction in which there is little 
evidence of student engagement with mathematical ideas. Level 2 describes instruction 
with elements of effectiveness that has substantial problems in the design, implementation 
or content of the lesson, and is limited in the likelihood to enhance children’s mathematical 
knowledge. Level 3 refers to beginning stages of effective instruction characterized by a 
few elements of effectiveness that frequently engage children in mathematical concepts, 
but has some weaknesses. Level 4 reflects effective instruction that is engaging for most 
students, whereas level 5 describes exemplary instruction that engages all of the students 
most of the time and represents the art more than the craft of teaching. 

In general, classroom observations in the baseline year revealed variability in the 
quality of mathematics instruction. Although a few of the observed teachers were already 
somewhat skilled at standards-based teaching practices, most teachers were not (see Table 
4). A couple of the observed lessons were ineffective, and several had only elements of 
effective instruction. For example, some lessons included children working with 
manipulatives, but the mathematical concepts central to the activity were hardly addressed 
and higher order thinking was minimal. The next section will describe several 
observations that reveal the quality of the instruction found during the observations. The 
two levels most frequently represented were beginning stages of effective instruction and 
elements of effective instruction. 

Table 4. Frequencies of Qualities of Mathematics Instruction in Observed Classrooms 
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Source : Classroom observation protocols 



Beginning Stages of Effective Instruction 

In several observations, teachers presented beginning stages of effective 
instruction , but the lesson was limited somewhat in its capacity to enhance the 
mathematical understanding of most children in the classroom. The following case 
characterizes the observed lessons that manifest this quality of instruction: 
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Using the bean toss activity from the book Math Their Way (1978), 
a kindergarten teacher helped children develop the concept of one- 
to-one correspondence, and learn groups of numbers that sum to 
10. In pairs, students tossed 10 beans (one side red, one side 
white) from a plastic cup, counted the number of red beans, and 
recorded the number with a red crayon on a data sheet with 10 
blank beans. The students were familiar with the procedure 
because in previous lessons they had counted and recorded data for 
smaller sums (i.e., 7, 8, 9). While children took turns tossing and 
counting the red beans, they confirmed and disconfirmed each 
other’s counting. For example, in one pair a boy tossed the beans 
and counted, “1, 2, 3, 4, 4, 5.” His partner said, “4, 4, 5?” She 
recounted the beans out loud and got 6. The boys said, “Oh 6!” 

As the pairs finished collecting their data, several students taped the 
data sheets together for each number up to 10 to make a histogram 
at the front of the class. These children talked to each other and 
analyzed the data they were gathering, for example, “I think 4 is 
winning,” “Now 5 has more.” In a large group, the teacher 
wrapped up the activity by discussing which numbers (e.g., 4 and 5) 
occurred most often. Thus, in this lesson, children were actively 
thinking and processing mathematics with one another. ■ One 
drawback is that the teacher did not manage the classroom to 
ensure that most of the children were actively engaged; some 
children dominated the pair work while others were sometimes off- 
task. 

Source: Classroom observation field notes 

This observation illustrates that some teachers are practicing standards-based 
mathematics instruction effectively even before they participate in ACME professional 
development. In this observed lesson, the teacher engaged kindergarteners in an 
investigative activity that involved exploring developmentally appropriate concepts in 
mathematics (e g., one-to-one correspondence, counting to 10). Yet, this observation 
presented areas that need improvement, specifically the teachers’ classroom management 
skills. Finding strategies to ensure that most of the children in the classroom are equally 
and simultaneously engaged, this teacher could become more effective at teaching 
standards-based mathematics. Other teachers whose lessons were rated in this category 
also could improve their effectiveness through the professional development provided by 
ACME. 

Elements of Effective Instruction 

The quality of the instruction in several of the observed classrooms was primarily 
ineffective, but included some elements of effective instruction. The following observation 
demonstrates the characteristics of this category of instruction. 
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A teacher had second grade children work with manipulates to 
form groups, rows, and stacks of snap cubes (e.g., 3 stacks of 4 or 
2 groups of 6) and to represent multiplication and division. On an 
overhead projector, the teacher presented a table with three 
columns: number of blocks in a set, type of set (i.e., groups, rows, 
or stacks), and number of sets. Each child had a copy of the table 
to keep track of the problems, and each group of four children 
whose desks were together had a box of cubes. The teacher called 
on children to choose a number and a type of set. She rolled a die 
to determine the number of sets. Each child then represented the 
sets with snap cubes, and the teacher walked around the room 
checking and correcting their representations. 

During the lesson, several children incorrectly represented the sets; 
the teacher corrected some but ignored others. She paid a great 
deal of attention to how children laid out their rows. The children’s 
desks were organized in groups of four. Children sitting next to 
each other looked at children’s arrangements, but rarely discussed 
the problem to make sense of it. Toward the end of the lesson, the 
teacher suggested that the children help one another represent the 
problems. A boy tried to help a girl correct her answer by telling 
her that she had 5 groups when she needed 6, but their box of 
manipulatives was empty and she was unable to correct her 
representation. This teacher also tended to ignore girls and call on 
boys. Moreover, the desks of three boys who had trouble following 
the assignment were grouped at the front of the room, but the 
teacher rarely visited their table to guide their thinking and actions. 

At the end of the lesson, the teacher abstracted the problems with 
symbols, but did not accept variability in children’s accurate 
responses to her questions. 

Source: Classroom observation field notes 

This case exemplifies instruction that has a few elements of effectiveness but does 
not appear to improve the mathematical competence of many children in the classroom. 
Some children appeared to benefit from representing multiplication and division with 
manipulatives and were able to link their concrete experiences to the symbolic abstraction 
at the end of the lesson. However, many of the children in the class did not. Their work 
reflected a lack of understanding, but the teacher did not successfully address their 
misunderstandings. Moreover, children’s own efforts to help one another were 
ineffectual, given their apparent inexperience with cooperative learning and the limited 
materials available for the lesson. This teacher’s instructional practices could be improved 
by acquiring standards-based strategies for classroom management and perhaps more 
content knowledge of mathematics. These teaching practices also would be more 
effective if all of the children in her classroom had equal access to instruction and 
resources. On the basis of classroom observations, a number of teachers observed lack 
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the instructional strategies necessary for implementing effective, standards-based 
mathematics. 

In addition to improving mathematics teaching and learning in general, district staff 
would like to explore how standards-based mathematics could improve the performance 
of students in special education. For the three special education lessons observed, the 
quality of the instruction contained elements of effective instruction. The following case 
illustrates characteristics of these lessons. 

A sparsely equipped classroom included five children, a teacher, 
and a teacher’s aide. One fourth grade child worked One-on-one 
with the teacher to practice multiplication by four with stacks of 
snap cubes. The teacher asked the child to count how many cubes 
he had stacked. After the child correctly counted the cubes, the 
teacher asked how many groups of four he had. The child gave the 
number, and the teacher stated, for example, “So 6 times 4 is 24.” 

These procedures were repeated for stacks of one to six cubes. 

Much of the interaction centered on focusing the child’s attention, 
but he was willing to work at times. Much of the teacher’s 
attention was spent constraining and monitoring the behavior of all 
of the children in the classroom while working with the target child 

Source : Classroom observation field notes 

In this observation, the teacher presented some elements of standards-based 
instruction such as work with manipulatives. However, the interaction was highly 
structured, and primarily the teacher, not the child, made the connection to multiplication 
from the groups of snap cubes. Although the teacher designed a lesson with concrete 
representations of mathematical concepts, other activities might have provided a richer, 
more interesting learning experience. Relinquishing some of the structure and adapting 
some of the activities from the curriculum resources may go further in enhancing the 
learning experiences and understanding of children in special education classes. In the 
baseline year, the ACME project staff have begun to help special education teachers plan 
ways to adapt the new curriculum and instructional strategies in their classrooms. 

In conclusion, although many of the untrained teachers report that their 
mathematics instruction includes investigative practice, direct classroom observations 
contradict their reports. Only a small number of teachers in the observed sample were 
familiar with standards-based practice and effectively engagd children in investigative 
activities. According to these ratings, some teachers who have not yet participated in 
professional development are not effectively teaching mathematics with the strategies 
outlined in NCTM standards. For the most part, the mathematics instruction in the 
baseline year of the project was ineffective or contained only elements of effective 
practice. 

Quality of ACME Professional Development 

The quality of professional development that the ACME facilitators provided was 
evaluated primarily from eight direct observations. All observations involved sessions 
with teachers, except one session with administrators. Feedback from teachers about 
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professional development came from two additional sources. First, teachers who 
participated in professional development responded to questionnaire items. Second, 
teachers who attended the summer institutes of 1998 completed evaluations about those 
experiences in professional development. At the end of the first year, the project had 
developed a strong professional development plan that supports implementation of 
standards-based curriculum and instruction and is responsive to participants’ needs. 

Observations of Professional Development 

The ACME project has planned long-term professional development and ongoing 
support for all mathematics teachers in the district in accordance with current staff 
development models, including yearly week-long institutes and continued support during 
the academic year. 

Structure of Professional Development 

The ACME project staff have developed what one staff member called “custom- 
designed” professional development. Project staff members have assembled materials and 
activities to promote exploration and acquisition of standards-based pedagogy, 
mathematical content and instructional strategies tailored to the preparation of the 
district’s teachers. The observed professional development sessions addressed classroom 
pedagogy by modeling effective mathematics instruction, demonstrating how to use 
materials in the classroom, and examining issues of scope and sequence in elementary and 
middle school mathematics. In a majority of the observations, facilitators promoted 
reflective practice. Topics that received little attention in the observed sessions included 
using computers in classrooms and addressing issues of access, equity, and diversity. 

In more than half of the observed sessions, the discussions and activities covered 
specific mathematics content. The project staff focused on broad topics that go beyond 
the computation and numeration traditionally taught in classrooms. Computation arose 
primarily as a tool for solving problems, not as a topic unto itself. The sessions involved a 
variety of topics such as numeration and number theory, measurement, patterns and 
relationships, and geometry. Additional topics informally observed included pre-algebra, 
data collection and analysis, and estimation. The topics of probability, statistics, and 
calculus were not observed. 

Project staff designed professional development to actively involve participants 
with their peers and to model investigative activities and instructional strategies. In the 
format of all professional development observations, the facilitator led large group 
discussions with the participants, and in all but one of these sessions, teachers discussed 
topics in small groups. In almost all observations with teachers, the participants were 
involved in investigative activities and problem-solving. In some observations, teachers 
briefly read NCTM standards or journal articles before discussing the topics in small and 
large groups. In only one observation, a session for school administrators, did the 
facilitators primarily present information to the participants. 

Quality Ratings of Professional Development 

As for the classroom observations of mathematics lessons, the evaluator rated the 
quality of design, implementation, mathematics content and culture of the professional 
development observed as well as the quality of pedagogical content. The observations 
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were assessed on a 5-point scale ranging from ineffective to exemplary professional 
development. Level 1 refers to ineffective professional development that does not engage 
participants in ideas of mathematics education and is unlikely to influence their capacity to 
provide high quality instruction. Level 2 contains elements of effective professional 
development such that participants are sometimes engaged, but the session has substantial 
problems and is limited in its positive impact on instruction. Level 3 presents beginning 
stages of effective professional development with activities that are sometimes effective in 
enhancing participants’ effectiveness as educators, but has some weaknesses. Level 4 is 
effective professional development that engages most participants in the concepts of high 
quality mathematics education and influences their capacity as leaders. Level 5 refers to 
exemplary professional development in which the facilitator demonstrates skill and artful 
orchestration of the session, and enhances the capacity to provide high quality 
mathematics education and leadership qualities of all participants. 

Table 5 presents the means and standard deviations across the 5-point scale for 
dimensions of the eight professional development observations. Overall, the quality of the 
ACME project’s professional development is effective, but some sessions revealed areas 
for improvement. In general, the strengths of the observed professional development lie in 
the design and the culture of the sessions, both of which averaged at level 4. The design 
included the activities, the roles of participants and facilitators, organization, and resources 
of the observations that influence the participants’ capacities to provide children with 
effective mathematics education. The culture involved a climate of collaboration, respect, 
and encouragement of teachers’ active participation in sessions. 

Table 5. Means and Standard Deviations of Dimensions of 
Eight Professional Development Observations 
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Source: Professional development observation protocols 

Other dimensions of the observed professional development sessions were 
effective but less powerful. Implementation, or the style in which the facilitator carries out 
the session, was variable (represented by a standard deviation of 1 .0). On the basis of this 
finding, implementation of some observations was exemplary professional development, 
but the implementation of others presented beginning stages of effective professional 
development. The dimensions of mathematics content and pedagogy had the lowest 
average ratings (Mean = 3 .4). It is possible that the variation in implementation is related 
to the lower ratings for mathematics content and pedagogy. If the facilitator is not highly 
effective at carrying out the professional development, the communication about 
mathematics content and pedagogy may be less likely to move participants’ understanding 
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forward. Comparison of the following two observations that represent quality ratings of 
level 4 and 3 illustrates this point. 

Effective Professional Development 

The following observation exemplifies highly skilled implementation that supports 
participants’ exploration of standards-based pedagogy. 

This session was the first day of the second summer institute for 
fifth grade teachers with about 40 participants. The half-day 
session had four parts. The session began with a journaling activity 
on participants’ thoughts and experiences with mathematics 
followed by problem-solving in small groups, a reading and 
discussion of four major NCTM standards of mathematics 
instruction, and a planning session. A highly engaging group 
discussion followed each of the first three activities. 

In this session, the facilitator skillfully and with energy engaged the 
participants in high level discussions of investigative mathematics 
and instructional strategies. She followed the ideas brought out in 
discussion, summed up what was said, and expanded the concepts. 

The atmosphere centered on respect for participants, punctuated by 
warm adult humor. During small group work, the facilitator visited 
every table to assess their progress and push their understanding 
forward. 

Through personal examples, the facilitator exchanged ideas with 
participants about pedagogy and shared expertise. For example, in 
a discussion of sizes of cooperative learning groups, she shared the 
struggle to find a workable number and welcomed participants to 
explore possibilities in their own classrooms. In another discussion 
of mathematics as reasoning, the group addressed strategies for 
comparing fractions. The facilitator described an instance when a 
child wanted help carrying out an algorithm that he did not 
understand. One teacher replied, “Doesn’t a strategy mean 
something that makes sense?” The facilitator said that she did not 
understand why the algorithm worked. Another teacher said, “If 
you don’t understand, why teach it?” Still another participant 
challenged the idea because that particular algorithm was the only 
way that she could get children to compare unequal fractions. 

Finally, one teacher in the group figured out the logic of the 
algorithm. 

Source: Professional development obsen’ation field notes 

This observation illustrates how effective facilitation actively involves teachers in 
professional development. In this example, teachers challenged traditional approaches to 
mathematics education, explored the meaning of standards-based strategies, and aired 
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different opinions. The skill level of the facilitator seemed to promote the engagement of 
participants and to boost their thinking about standards-based pedagogy to high levels. 
The effective implementation of the session’s design and rich cultural climate supported 
the deep exploration of pedagogy. 

Beginning Stages of Effective Professional Development 

The next example illustrates how implementation that is not fully effective may 
hamper the exchange of information between facilitator and participants about 
mathematical content. 

The facilitator led two activities in a room with 50 fourth grade 
teachers. First, in small groups, the participants were given eight 
cards, each with a different competency level in fractions that 
children should demonstrate at different grades. The groups 
compared these cards to the state standards (TEKS) to determine 
to which grade level each competency referred. The next activity 
involved exploring fraction bars. 

This facilitator fostered a warm climate of collaboration and 
exploration, for example, with jokes and by asking participants if 
they agreed with the assertion of another. Several times the 
facilitator made one-way presentations to the group about topics 
such as the difficulties of coming to a consensus, student mobility in 
the district, and not accepting documents at face value. The 
activity with fraction bars was limited to some comparisons and a 
few points about the properties of the materials rather than 
examining multiple activities with the bars and discussing how to 
support children’s mathematics understanding through the activity. 

Source: Professional development obser\>ation field notes 

This observation demonstrates how limited implementation may detract from the 
participants’ exploration of mathematics content. The facilitator could have encouraged 
participants to discuss their expertise and delved more into the development of children’s 
thinking and mathematical concepts embedded in fractions. This facilitator also did not 
draw out mathematics pedagogy as in the previous example. 

In conclusion, the ACME professional development highly involved participants in 
investigative activity, the exploration of pedagogy, and the acquisition of mathematics 
content. Overall, the ACME project staff provided effective professional development, 
although some sessions were somewhat limited in the dimensions of mathematics content 
and pedagogy. It is possible that by strengthening the implementation skills of some 
facilitators, the project will improve the mathematics content and pedagogy provided and 
enhance teachers’ capacity to implement standards-based curriculum and instruction. 

Teachers’ Attitudes Toward ACME Professional Development 

Teachers who had participated in ACME professional development reported on 
their attitudes and experiences. However, these responses must be interpreted with 




25 



97.18 



ACME. 1997-98 Report 



caution because it appears that many teachers may have misunderstood the items. An 
introduction to these questions referred to “the NSF-supported Local Systemic Change 
(LSC) project.” The acronyms LSC and ACME were explained in a cover letter that 
accompanied the questionnaire. However, the responses indicated that teachers may have 
thought that the items referred to any mathematics professional development in the 
district. For example, over 50% of the teachers who responded indicated that they had 
received 20 hours or more of LSC professional development. Yet, according to 
attendance records as of March of 1998, only 14% of the teachers who received 
questionnaires had attended 20 hours or more of ACME professional development. In an 
earlier survey, teachers reported that they did not recognize the name of the project as 
ACME, and in the first year they often referred to the project by the names of the 
curriculum resources. Investigations and CMP (Batchelder, 1998). Moreover, prior to the 
ACME summer institutes of 1 998, many teachers did not know that NSF has funded the 
ACME project, and were probably less familiar with the acronym of LSC than ACME. 
Because most of the teachers who responded to the questionnaire were not fifth or sixth 
grade teachers, and many had received professional development throughout 1997 and 
1 998, the validity of these responses is questionable. 

Teachers who reported that they attended “LSC” professional development were 
asked about the extent of their involvement. In general, a majority of these teachers gave 
intermediate responses between involved not at all and to a great extent. Over three- 
quarters of these teachers perceived some encouragement to develop an individualized 
professional development plan for mathematics education. Three-quarters or more of 
these teachers felt supported with time to work with other teachers in professional 
development, time to reflect on classroom applications, and help in implementing what 
they learned. In contrast, 29% of the mathematics teachers and 41% of the special 
education teachers who responded did not feel at all involved in planning their own 
professional development in mathematics education. In summary, many teachers who 
responded felt some encouragement and support in their professional development 
regarding mathematics education, although a number of these teachers did not feel 
involved at all in the planning. 

When asked how the “LSC” professional development increased their knowledge 
and skills in mathematics, again, most teachers gave responses between not at all and to a 
gi A eat extent. Over 80% of the teachers reported that participating increased their 
mathematics content, their understanding of how children think about/learn mathematics, 
and their ability to implement high-quality mathematics instructional materials. Overall, 
more than half of these teachers rated the “LSC” professional development as fair or 
good, over one-fifth rated it as very good or excellent, and less than one-fifth rated it as 
poor or very poor. 

In sum, for most of the teachers who responded to these items, their feelings about 
the professional development in mathematics education are between positive and negative. 
As a group, they reported neither strong negative feelings nor strong positive feelings 
about the project and their participation in charting their own professional development in 
mathematics education. Nevertheless, because many of these teachers may have 
misunderstood that these items referred only to ACME professional development, it is 
likely that these findings do not paint a representative picture of how teachers feel about 
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ACME professional development. These responses possibly included the teachers’ 
thoughts about other mathematics professional development in the district. 

Key Features of ACME Professional Development 
Curriculum Materials 

The ACME professional development provides teachers with a rich opportunity to 
explore and become conversant with the curriculum resources of the project, 
Investigations in Number, Data, and Space and Connected Mathematics Project, in a 
number of ways, such as modeling lessons and analyzing units above and below grade 
levels. The project staff also helped teachers become familiar with pacing of units, 
demonstrated how choice time works, and how to implement partner quizzes. Participants 
solved problems using nontraditional algorithms, modeled mathematics concepts, and 
shared strategies for approaching mathematics. 

The following example demonstrates how facilitators lead participants in hands-on 
experiences with specific activities found in the resources. 

In one observation, participants picked up materials for four games 
from Investigations that involved work with whole number 
computation and fractions: Close to 100/Close to 1000, Double 
Compare, The Fraction Cookie Game, and Multiple Bingo. They 
were told to play each game for 10 minutes and to think about the 
important mathematical ideas behind the games, modifications, 
additional questions to ask students, and suggestions for colleagues. 

At the end of the activity, the large group discussed participants’ 
reflections. 

Source: Professional development observation field notes 

By sampling the games, participants learned logistically how to manage the same activities 
in their classrooms. They practiced communicating about the materials in small and large 
group discussions and developed a deeper understanding of the implications for practice 
and the mathematical ideas behind the games. In the particular observation cited, several 
teachers came up with variations on the games to further children’s mathematical 
exploration through the activities. Thus, teachers were supported as they tested out 
materials and developed skills in communicating with colleagues about the curriculum 
resources. 

Additionally, the ACME project staff supported teachers’ exploration of the 
designated curriculum resources by addressing their requests. During the 1997-98 school 
year, fifth and sixth grade teachers requested time to compare the curriculum resources 
across grade levels. ACME project staff thus let participants decide what the last follow- 
up day would look like and designed materials to guide discussions of the similarities and 
differences in the fifth and sixth grade curriculum resources. Thus, in this example, 
ACME project staff supported not only teachers’ exploration of materials, but also 
responded to participants’ needs to understand scope and sequence in the project’s 
mathematics curriculum across resources and grade levels. 
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Mathematics Content 

A major thrust of the ACME project staffs design of the professional development 
in the 1997-98 school year was to help teachers deepen their knowledge of mathematics 
content. Major areas of focus included fractions, decimals, and multiple strategies for 
approaching mathematical operations (i.e., addition, subtraction, multiplication, and 
division). Some units on geometry and algebraic concepts were included. Estimation and 
measurement were sometimes covered as secondary topics. Facilitators also used “ten- 
minute math” exercises to demonstrate data collection and analysis. 

In content areas where the project staff recognized that the designated curriculum 
resources had gaps at the fourth grade level (e.g., in decimals), the professional 
development provided participants with supplementary exercises and materials. Through 
group activities, project staff introduced many tools that support these content areas such 
as fraction cubes, bars, and blocks; snap cubes; tens and ones blocks; and so on. Some 
topics such as geometry, measurement, and probability and statistics, were infrequently 
addressed and could be expanded in future years. 

In general, the ACME professional development in the baseline year exposed 
participants to often neglected areas of mathematics content and helped teachers move 
beyond the traditional mainstay of computation. The facilitators presented different 
models for approaching mathematical problems such as drawing pictures. In addition, the 
project staff facilitated teachers’ developing their abilities to “do” mathematics in ways 
they had never before. The following example reported by ACME project staff 
demonstrates how one teacher attained algebraic thinking through a professional 
development activity. 

In small groups, teachers translated data from tables to graphs and 
talked about the relationships between the two formats. They used 
the numbers in the tables to describe the relationships in words. 

Then they expressed the data in equations. At the end of the 
exercise, one teacher reported that he had never understood before 
how people got equations from numbers and patterns. 

Source: Project staff interviews 

This example illustrates how a teacher gained the ability to carry out a mathematics 
procedure that he had not been able to do before from the ACME professional 
development. This teacher acquired knowledge and skills that he can now take back to his 
classroom and share with his students. 

Pedagogy 

A strong point of the ACME professional development is the depth and complexity 
in its approach to standards-based pedagogy. This point is powerful because central to 
standards-based practice is an understanding of the underlying philosophy and how this 
practice differs from traditional practice. As mentioned previously, in the first year, 
project staff recognized that most teachers did not understand NCTM standards for 
mathematics education. In the summer institutes of 1998, the staff designed an activity for 
all participants — newcomers and trained teachers alike — to reflect on and come to 
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consensus with their peers about the meaning of mathematics as problem-solving, 
communication, reasoning, and connections to other disciplines. Other topics addressed 
included asking questions to promote children’s mathematics competence, journal writing 
to encourage reflection, and transforming ideas of reforms in mathematics education. 

Assessment 

The standard s-based assessment strategies that accompany this pedagogy is an 
area of major concern for teachers. Grading and assessment emerge often in participants’ 
questions to facilitators and evaluations of ACME professional development. A major 
struggle for teachers has been adjusting their grading to performance-based assessment. 
As noted only one-quarter of the teachers who responded to the questionnaires reported 
that their students took tests that required descriptions or justifications, which are 
characteristic of investigative practice, often or always (see Table 2 in Classroom 
Instruction in the Baseline Year). As a result, teachers who have not received professional 
development in how to develop such assessments may be at a loss for how to proceed. In 
ACME follow-up professional development in the Spring of 1998, an example of how 
ACME project staff is supporting teachers in their struggle with assessment occurred 
when fifth grade teachers brought in student work that they had evaluated with rubrics. A 
rich discussion of students’ reasoning, problem-solving strategies, and mathematical 
understanding ensued. To address teachers’ concerns, project staff should expand and 
allocate more time for this sort of activity in future follow-up sessions. 

Generally, in the first year of the ACME project, the professional development 
provided participants with opportunities to explore and deepen their knowledge of the 
designated curriculum materials, mathematics content, and effective pedagogy. The 
project has some room for improvement in various areas. For one, some facilitators 
implement the professional development more proficiently than others. Taking steps to 
strengthen the skills of staff who are less experienced than others could not only improve 
implementation but also the delivery of mathematics content and pedagogy. Another area 
of improvement could be to build on the basic mathematics content already addressed with 
more diverse topics. Assessment is another area to consider expanding and devoting more 
time to in follow-up sessions. 

Support for Teachers’ Implementation 

By design, ACME professional development supports teachers as they try out the 
curriculum materials in their classrooms. As noted, the professional development plan is 
extensive and intends to involve each teacher for three years. In addition to two years of 
summer institutes and follow-up professional development during the school year, 
teachers participate in one year of support on their campuses. These follow-up sessions 
provide opportunities for teachers to bring in their experiences from their classrooms and 
share with their peers. Teachers also exchange strategies that they used for classroom 
management, assessment, student grouping, differentiation, and problem-solving. For 
example, project staff reported that teachers brought in factor pairs for the number 1 1 00, 
looked at all of the different strategies, and compared them. Although the project team 
orchestrated this activity, the team stepped back and the participants took over by pointing 
out how other teachers’ students were approaching the problem. 
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ACME project staff have adopted other strategies to support teachers in 
implementing the new curriculum materials. A substantial area of support for 
implementation was allotting teachers planning time during professional development and 
providing an opportunity for them to share with their colleagues. On the teacher 
questionnaires, 67% of the mathematics teachers and 77% of the special education 
teachers who responded reported that they did not have time during the regular school 
week to work with their peers on mathematics curriculum and instruction. In evaluations 
of professional development and in direct conversation with facilitators, participants 
requested time to plan with their peers. Thus, project staff integrated this activity into the 
plan of professional development sessions. In addition to planning time to support 
implementation, project staff laid out suggestions for the order of units and number of 
days to spend on each unit as well as basic structures for mathematics lessons. Project 
staff also supplied participants with supplementary materials to carry out activities in their 
classrooms, such as card stock copies of activity pieces from the resources. 

In addition to support in professional development, ACME project staff are 
beginning to provide teachers with more personal, informal support. In the Fall of 1998, 
the ACME project staff began devising a method for team teaching with teachers in the 
district. The purpose of this plan is to provide campus-based support and to allow staff to 
return to classrooms, but also to develop strategies to support teachers who are 
attempting to implement the standards-based curriculum and instruction on their 
campuses. Because the project staff consists of eight mathematics specialists who cannot 
reach personally the more than 2000 mathematics teachers who will participate in ACME 
professional development, the project must enlist more facilitators. Perhaps staff could 
adapt this model of peer coaching to teachers and develop strategies to support more 
teachers districtwide. 

Quality Facilitation 

At the end of the first year of the ACME project, the professional development 
facilitators consisted of eight mathematics specialists. Project staff reported that all team 
members were hired because they were outstanding mathematics teachers, demonstrated 
leadership on their campuses in mathematics education, and implemented a standards- 
based curriculum in their classrooms. Every one had experience providing professional 
development to other teachers through the district, regions of the state, and/or national 
organizations, although some had more experience than others. Additionally, every team 
member developed their own professional development by attending conferences and 
workshops where they received training and information that they brought back to the 
classroom. 

In the first year, the ACME project essentially had no teacher leaders to facilitate 
professional development. At that time, the project staff was able to handle the demand. 
This plan was devised in collaboration with the Dana Center to avoid the diffusion of 
knowledge and skills that can occur when information passes through multiple trainers 
with different levels of expertise. However, to continue the project as planned, ACME 
project staff recognize that teacher leaders will need to be added to help with professional 
development in the Summer of 1999 because the number of teachers attending will 
increase as grade levels are added. As the cross-site evaluation of local systemic change 
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suggests (Weiss, Rapp, & Montgomery, 1997), the quality of professional development 
that teachers receive may decrease. ACME project staff should take steps to ensure that 
teacher leaders are well trained for their roles and that they are motivated to become 
leaders in mathematics reform. 

Dei’elopment of Professional Learning Community 

Several components of ACME professional development foster a learning 
community of professionals. The ACME project staff of eight members together comprise 
a community of learners. As one staff member stated, the team has a “focused vision,” use 
the “same language,” and is “philosophically aligned.” The team has a culture of learning 
and supportive collegiality. The team’s vision and enthusiasm for learning spreads to 
participants in professional development. The team encourages participants to build on 
the community by presenting itself as an accessible resource and by drawing upon the 
experiences of teachers to facilitate rich discussion during professional development. 

One facilitator opened a summer institute with an invitation for participants to 
interrupt and an impromptu question and answer period. This climate inspired participants 
to share and support each other. Another observation of fourth grade teachers’ summer 
institute began in the following way: 

Before a planned activity, one teacher stated, “Sometime before we 
leave, I’d like to hear somebody who’s used Investigations talk 
about how to use it.” The facilitator suggested a panel discussion, 
but then took a secondary role as teachers launched into a 45- 
minute exchange about their experiences using the curriculum 
resources. [Although fourth grade teachers are scheduled to begin 
implementation in the 1998-99 academic year, some schools have 
forged ahead of the ACME project’s plan] 

What ensued was a rich exchange between participants about the 
trials and tribulations of trying out the curriculum. One teacher 
said, “I trained students in cooperative strategies.” Another said, 

“One thing that was very difficult is having kids talk. .. You build 
on it every day.” Still another shared, “I started at the beginning of 
the year. I decided to do everything as in the book. By January, I 
was convinced this was better than what I did before.” Teachers 
mentioned letters in the back of the book to send to parents to 
explain the curriculum as well as how children picked up alternative 
problem-solving strategies from each other. The open conversation 
continued as teachers new to the curriculum asked others how they 
handled substitutes, homework, and differentiation for children with 
different levels of competence. 

Source: Professional de\ j elopment observation field notes 

This observation illustrates how the environment that the ACME facilitators have 
established for professional development can set the stage for a powerful exchange 
between adult learners. In this example, the community of learners ignited spontaneously 
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by sharing essential information and supporting one another. The ACME project provided 
a forum in which teachers themselves addressed concerns about implementing the new 
curriculum and instruction. To further guide the development of a professional 

community of learners and to foster ownership in the community, ACME project staff 
need to recognize similar events as they emerge. 

Another format for developing a community of learners in the original design of 
the ACME project was to foster support among teachers on every campus. Thus, in the 
first year of the ACME project, grade level teams attended follow-up sessions during the 
school year together. This design allowed campus teams to take their shared learning 
experiences from professional development back to their campuses and expand on them. 
Although many teachers appreciated this arrangement, some teachers and school 
administrators did not feel comfortable with all teachers from one grade level being away 
from their campuses on the same day. To accommodate this concern, ACME project staff 
planned a variety of follow-up options, including release time during the day and afternoon 
and Saturday sessions. These options allow campuses to continue to attend together. 

In sum, as ACME project staff support implementation and contribute to the 
development of a professional learning community, the strengths emerge in the flexible 
character of the staff and their willingness to take stock and make adjustments in the plan. 
However, one point to consider is the size and scope of the project. In a growing urban 
district, ACME project staff cannot address every need or concern that arises. In the 
team’s August retreat, a major theme that arose was sharing responsibility for the project. 
Teachers, parents, administrators, central office staff, board members, and other 
stakeholders in the community can also take a lead to help make the project work. Project 
staff needs to take note of where initiative and motivation may be emerging across the 
district. While the staff could remain a resource for support and ideas, recognizing 
interest, enthusiasm, and devotion to the new curriculum and resources could help share 
the responsibility for making the project work. 

School and Teacher Involvement 

The level of school and teacher involvement in the ACME project is generally 
proceeding on schedule. Most of the first cohort of fifth and sixth grade teachers have 
participated in two summer institutes and follow-up professional development sessions 
during the 1997-98 school year. In the summer of 1998, the second cohort of fourth and 
seventh grade teachers attended their first institute, and many school administrators 
attended sessions to promote campus-based leadership and support for the project. In 
addition, the ACME project staff have encouraged involvement by keeping teachers and 
administrators informed of professional development schedules through letters and 
brochures. However, the level of school and teacher involvement in the ACME project’s 
first year has not been complete, which may be due in part to the difficulty of reaching all 
of the possible participants in a project of this scope. 

Teacher Involvement 

In general, all 83 elementary and middle schools in the district, except one school, 
have teachers who have participated in some ACME professional development. In 
addition, most teachers from the targeted grade levels have participated in some ACME 
professional development. Although district employee records are not always accurate 
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because teachers’ positions change from year-to-year and changes are not always updated, 
estimates support a high level of involvement. Out of approximately 60 sixth grade and 60 
seventh grade mathematics teachers, attendance at the summer institutes of 1998 was 
nearly 100% of the teachers. Out of approximately 250 fourth grade teachers, attendance 
was about 90% of the teachers for the first week and about 80% of the teachers for the 
second week of their first summer institute. Similarly, out of approximately 270 teachers 
from pilot schools, 90% of the teachers attended their first summer institute. However, 
out of approximately 250 fifth grade mathematics teachers, approximately 40% attended 
the second summer institute. 

The low attendance rate for fifth grade teachers in the summer of 1998 could be 
explained by departmentalization. In up to 20% of the district’s elementary schools, fifth 
grade teachers teach either mathematics and science or language arts and social studies. 
Yet, departmentalization does not completely account for this low attendance rate. 
Project staff reported that some fifth grade teachers reported that they did not attend the 
second summer institute because they “were already trained.” Because the ACME project 
staff do not want this belief to persist, staff members have taken steps, such as calling and 
visiting schools, to clear up misunderstandings about schedules and the ACME 
professional development design. Consequently, enrollment of fifth grade teachers for the 
first follow-up of their second year in the project is up to about 75% of the teachers. 

All of the teachers who participate in the ACME project teach at schools that serve 
a diverse population of students. In the principal questionnaire, school administrators 
reported that racial and ethnic make up of students in the district’s schools were on 
average 46% Hispanic, 35% White, 20% African American, 2% Asian or Pacific Islander, 
and 0% American Indian or Alaskan Native, although the composition of each school 
varied across campuses. Principals also reported that 19% of the students had limited 
English proficiency and 61% were eligible for free or reduced lunches on average, 
although this percentage also varied across campuses. Thus, the ACME project is 
reaching schools with diverse ethnic/racial backgrounds, language proficiency, and family 
socioeconomic status. 

The K-8 mathematics teachers who are targeted to participate are primarily female 
and White or Hispanic. Among the teachers who responded to the questionnaire, 95% of 
the mathematics teachers and 90% of the special education teachers were women and 5% 
of the mathematics teachers and 10% of the special education teachers were men. The 
racial or ethnic backgrounds of the mathematics teachers and special education teachers 
were, respectively, 61% and 76% White, 27% and 11% Hispanic, 9% and 5% African 
American, 1% and 5% Asian, and 2% and 3% with other backgrounds. One-third of the 
mathematics educators had taught for five years or fewer, 18% for 6-10 years, 30% for 
11-20 years, and 19% for 21 years or more. A larger proportion (50%) of special 
education teachers were had taught for five years or fewer. Seventy percent or more of all 
teachers who responded had completed three or more semesters of mathematics courses in 
college. 

Principal Involvement 

Principal involvement in the project is in its seminal stages. When the principal 
questionnaire was distributed in the Spring of 1998, a few principals were not familiar with 
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the ACME project and did not respond to questions about it. Just 39% of the principals 
who responded to these items reportedly were familiar with the project to a great extent. 
Some principals (29%) reported a high level of involvement with the ACME project, 
whereas about the same number (25%) reported that they were not at all involved. 

In the summer of 1998, the ACME project staff began developing the campus 
leadership component that was part of its original plan. In the first year, project staff 
noticed that teachers were more knowledgeable about standards-based curriculum and 
instruction than were campus administrators. To address this imbalance, project staff 
planned three different two-day sessions for administrators to attend to learn about the 
new mathematics program, suggestions for leading teachers in implementation, and 
address parent and community concerns. After low attendance at the first session, project 
staff sent letters to administrators who were scheduled to attend the remaining sessions as 
reminders. At the beginning of the school year, because so few administrators had 
attended two days of the sessions, the deputy superintendent again required those who had 
not completed the sessions to attend make up sessions. 

Although a few principals in the district may be knowledgeable of NCTM 
standards and what standards-based instruction should look like, for the most part 
principals in the district are just beginning to learn about them. An observation of the 
second professional development with administrators revealed that their concerns centered 
on organizing materials rather than on deepening their knowledge about curriculum and 
instruction. In response the facilitator’s request for questions or concerns about the 
standards, principals asked about buying student books that accompany Investigations, 
preparation of materials in Spanish, and the delivery of materials to campuses. As the 
session continued, discussion returned to planned topics such as how the new curriculum 
and state standards are aligned, what standards-based curriculum looks like, and how to 
address parent concerns. 

In conclusion, school and teacher involvement is high, but transitioning. Nearly 
every school in the district has teachers who have attended ACME professional 
development and those schools serve a diverse populations of students. A majority of the 
teachers who are targeted to attend ACME professional development have participated, 
although in the second summer session the attendance of fifth grade teachers tapered off. 
Principal leadership in the project is in its seminal stages, and will require continued effort 
to increase and maintain the level of involvement. 

Supportiveness of District and School Contexts 

Central to NSF’s support for Local Systemic Change initiatives is the assumption 
that enhancement of curriculum and instruction does not take place solely with teachers 
and in classrooms. State and district policies, parents, principals and other forces that 
impinge on classroom instruction should influence teachers’ attempts to provide 
standards-based teaching and learning to all students. At the end of the first year, the 
ACME project was transitioning toward a supportive context with some policies and 
practices aligned with its vision and efforts underway to alter major barriers. 

Support for Teachers from Colleagues, Principals, Parents, and the Community 

In general, teachers in the district are collegial and prepared to support one 
another and work together for mathematics reform. Most teachers (86% of mathematics 
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teachers and 73% of special education teachers) who responded to the teacher 
questionnaires reported that they felt supported by their colleagues to try out new ideas in 
teaching mathematics. A majority of all teachers (over 60%) who responded felt that 
teachers in their schools had a shared vision of effective mathematics and regularly share 
ideas and materials related to mathematics. Although for many of the district’s teachers 
collegiality was high, time to work together was not a readily available resource. Less 
than one-fourth of all teachers who responded felt that they had time during the regular 
school week to work with peers on mathematics curriculum and instruction. 

Teachers who responded generally felt that their principals supported their efforts 
to improve mathematics education and collaborate with their peers on innovations, 
although special education teachers perceived somewhat less principal support than 
general education teachers. Most teachers (87% of mathematics teachers and 77% of 
special education teachers) felt supported by their principals to try out standards-based 
mathematics curriculum and instruction. Less half of the teachers who responded reported 
that their principal provided teachers time to meet and share ideas together, and some 
(40% of mathematics teachers and 30% of special education teachers) felt that their 
principal encouraged them to observe exemplary mathematics teachers. Moreover, a 
majority of the principals thought that these resources supported teachers. To encourage 
effective instruction, 67% of the principals supported giving teachers time for planning and 
preparation, 81% endorsed opportunities for teachers to work together, and 90% 
supported time for teachers to attend professional development. Thus, most teachers feel 
that their principal supports their efforts to adopt standards-based teaching practices, and 
most principals believe that the resources available influence the quality of instruction. 
However, a minority of teachers feel that their principal provides the resources necessary 
to successfully implement standards. 

In general, campus administrators responded to the principal questionnaires with a 
great deal of support for standards-based curriculum and instruction. Like teachers, 
principals highly endorsed investigative instructional strategies. Most principals (83% or 
more) considered investigative instructional strategies, such as cooperative learning 
groups, hands-on and inquiry-oriented activities, and performance-based assessment, very 
important for mathematics education. Almost all of the K-8 principals (89% or more) 
reported that they were knowledgeable of national standards for mathematics education, 
well-prepared to support their teachers in implementing the standards, and could accept 
the noise that emerges from an active classroom. As discussed in the previous section, 
many principals are just becoming knowledgeable of the full implications of these 
strategies for teaching and learning. The positive attitudes reported suggest that as 
principals learn more about standards-based mathematics, many could become a powerful 
force in the districtwide implementation. In interviews, ACME project staff 
acknowledged the many pulls on principals’ time that could distract some from fully 
supporting the project on their campuses. 

Parents reportedly provided teachers little support in school and classroom 
activities, and special education teachers received much less parental support than did 
mathematics teachers. Most teachers (32% of mathematics teachers and 63% of special 
education teachers) who responded reported that few or none of their students’ parents 
attend PTA meetings and family math nights. Although many teachers (61% of 
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mathematics teachers and 84% of special education teachers) reported that few or no 
parents voice support for investigative approaches to mathematics instruction, a similar 
number of teachers (47% of mathematics teachers and 66% of special education teachers) 
reported that few or no parents voice support for traditional approaches to mathematics 
instruction. Thus, teachers, particularly special education teachers, do not perceive much 
support or opposition to any kind of mathematics instruction from parents. Yet, in 
professional development teachers recounted stories about how a few parents challenge 
the standards-based mathematics curriculum and instruction. 

Support for the ACME project from parents and teachers is mixed as one ACME 
staff member reported. In the fall of 1997, after fifth and sixth grade teachers had 
attended the first summer institute, staff members attended a parent night at two different 
schools. At one school, teachers had garnered the parents’ support against the project, 
whereas at the other school the project staff was invited to provide information about and 
guidance for the project. These two examples suggest that support for reform in 
mathematics education is in transition and involves both barriers and facilitation. 

From the point of view of the ACME project staff, attitudes and beliefs of 
teachers, principals, and central office staff that support reforms in mathematics education 
are present but not quite ideal. The support from district staff is often verbal, but not 
action-based. Support from the community is more mixed than it is within the district. 
Although some campuses have been quite successful at engaging the community in 
meaningful ways, others are just beginning to recognize the importance of this source of 
support. The ACME project’s connections with institutes of higher education and other 
community stakeholders are cohesive at some points, but at not other points. The role of 
this external support is usually advisory. 

Thus, from this evidence, it appears that more teachers would be better able to 
implement standards-based curriculum and instruction with increased support from 
colleagues, principals, parents, and the community. Special education teachers report 
even less principal and parent support than general education teachers. Designing 
intervention strategies to buoy up these points of support is a challenge facing the ACME 
project staff. 

District Policies and Practices 

Some of the strongest supports of the ACME project are district policies and 
practices, including the quality of instructional materials, system for purchasing and 
managing materials, curriculum scope and sequence, and alignment with other reforms. 
From the perspective of the ACME project director, the district clearly supports the 
ACME project with materials: “The bottom-line is that no expense is too great.” The 
district is purchasing high quality materials and curriculum resources for all of the 
district’s schools and for all special needs, such as bilingual, ESL, and special education 
classrooms. The district’s adopted mathematics curriculum as well as the state standards 
the Texas Essential Knowledge and Skills (TEKS) are all aligned with national standards. 
Moreover, the reforms in mathematics are aligned with the district’s efforts in literacy and 
science education. 

On the other hand, the statewide and djstrictwide methods of assessment, have a 
mixed impact on the efforts to reform mathematics in the district. The state accountability 
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test the Texas Assessment of Academic Skills (TAAS) is clearly high stakes. Schools are 
rated according to student performance on the TAAS in reading, writing, and 
mathematics, and local newspapers publish the results of these ratings. The test consists 
of word problems with multiple choice responses and is not directly aligned with the 
performance-based assessment of national standards for children’s mathematics 
competence. The ACME project director reported that while some teachers in the district 
are interested in performance-based assessment, TAAS is the only means of assessment 
for others. The TAAS test puts a great deal of pressure on teachers so that their 
experimenting with other forms of assessment raises anxiety. The ACME staff reported 
that they knew that teachers would have questions about how standards-based curriculum 
and instruction prepare students to perform well on the TAAS. Consequently, in 
professional development, ACME staff helped teachers understand that the test is 
designed around the TEKS state standards and meets the expectations of TAAS. 

Thus, although district practices and policies tend to facilitate the ACME project, 
the state and districtwide assessments put a stress on full implementation of standards- 
based mathematics curriculum and instruction. 

Institutionalization of Professional Development System 

Whether high quality professional development will continue in the district after 
the ACME project ends is a question of institutionalization. In its first year, the project 
has developed an emerging infrastructure for a high quality professional development 
system in the district. Many components of the system are in place, and efforts to modify 
other components are underway. 

A major strength of the ACME project is that it fits into the district’s current 
system of professional development but expands and develops the system. A hindrance to 
the ACME project is that its staff provide professional development on top of their other 
duties as the district’s mathematics specialists. Similarly, teachers who would be leaders 
in the project would have to add leadership responsibilities to their work as teachers. 
Time and financial constraints thus limit the capacity of ACME staff and teacher leaders to 
provide high quality professional development to the district’s teachers and administrators. 
Nonetheless, the quality of professional development that the ACME project currently 
offers sets a precedent for other projects in the district to follow. 

The merging of other resources available to support ongoing, high quality 
professional development is in a transitional phase. Recently, central office administration 
has attempted to open communication about aligning the expenditure of federal and 
district funds with district policy including the standards-based mathematics curriculum 
and instruction. These efforts initially were met with both enthusiasm and opposition. 
Through these efforts, tension emerged between the district policy of campus-based 
management and other district mandated policies such as the ACME initiative to reform 
mathematics education. Simultaneously, principals receive authority to manage their own 
campuses and a mandated project to change mathematics curriculum and instruction. 

The structures to sustain high quality professional development are in the initial 
stages. Incentives for all K-8 mathematics teachers to participate in ongoing professional 
development at the school and district level and for all new K-8 mathematics teachers to 
be oriented to standards-based curriculum and instruction are being refined. The central 
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office staff have recently launched an evaluation to assess and improve professional 
development that the district provides, and examine the links between professional 
development and classroom implementation. Although the structures for high quality 
professional development are in place, there is considerable room for improvement. 
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Summary and Recommendations 

This report presented information about the progress of the ACME project in its 
baseline year. It pulls together information from the perspectives of mathematics teachers, 
principals, program staff and district administrators as well as from observations of 
classrooms and professional development sessions. At the end of the first year, the 
ACME project evaluation revealed strengths that will propel efforts to bring high quality 
standards-based mathematics education to all children in the district and challenges that 
need to be addressed to promote districtwide implementation. 

Strengths of ACME 

The major strengths of the ACME project in the baseline year include the 
following: 

• Most teachers who have not yet participated in ACME professional 
development and school administrators are familiar with the language of 
investigative mathematics and highly endorse its teaching practices. 
Consequently, the eagerness of some teachers and schools to participate in the 
ACME project may provide momentum for implementation throughout the 
district. 

• During the ACME project’s first year, project staff reported a transformation 
in teachers’ attitudes toward investigative teaching practice among those who 
participated in its summer institutes and follow-up professional development 
sessions. Teachers’ attitudes changed from anxiety, apprehension, and caution 
to interest and confidence in standards-based practice. Continuing to support 
this sort of transformation with teachers who are new to the project should 
further encourage teachers to try to implement the standards. 

• A strong point of ACME professional development was the extensive and 
primarily effective facilitation provided, particularly creating the investigative 
culture and design of the observed sessions. Across the board, in the first year, 
facilitators successfully modeled strategies that teachers could adapt in their 
classrooms and provided a rich collegial environment in which teachers could 
explore standards-based curriculum and instruction. 

• The ACME team forms a community of learners with a shared vision and value 
for quality mathematics education. This energy stimulates teachers and 
encourages their collaboration, within and across campuses. The team’s effort 
to correct issues as they arise constantly further renders the ACME project 
flexible and in tune with teachers and administrators in the district. The team’s 
responsiveness to teachers’ concerns and needs as they try out the standards- 
based curriculum and instruction allows teachers to modify their own 
professional development. 

• A major strength of the ACME project is the alignment of district policies and 
curriculum as well as the financial and material backing for the project. 
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Challenges of ACME 

The ACME project in its baseline year faces the following challenges in promoting 
implementation districtwide: 

• Teachers’ and principals’ familiarity with the language of investigative 
mathematics may conceal a lack of deep understanding of NCTM standards 
and implications for effective practice and students’ mathematical competence. 
Although teachers and district administrators may communicate in the 
language of mathematics reforms, their practice may not reflect the same level 
of understanding. One of the ACME project’s challenges is in determining 
where more understanding is needed. 

• Teachers who had not participated in ACME professional development 
described their classroom instruction in the baseline year as including both 
investigative and traditional practice, which may indicate incomplete standards- 
based instruction. Special education teachers reported somewhat less 
investigative practice and activities in their classrooms than did mathematics 
teachers. Although a majority of teachers characterized their teaching 
practices as including investigative strategies, observations of a random sample 
of mathematics classrooms in the district revealed instruction that was 
primarily ineffective as defined by Horizon Research Inc. with elements of 
effective practice and, in a few cases, effective instruction. The project’s 
challenge is to help as many teachers as possible understand and put into 
practice effective standards-based strategies. 

• Another challenge of the ACME project is to raise the quality of 
implementation, mathematics content, and pedagogy that all facilitators 
provide to the level of effective professional development. This task may pose 
a serious challenge as the project adds teacher leaders who may have less 
experience and fewer skills than current ACME staff. 

• Maintaining teacher involvement through the years of planned professional 
development is also a challenge of the project, particularly because the staff 
development literature shows that it takes several years for teachers to make 
new teaching practices their own. Additionally, ACME project staff need to 
gamer full involvement and leadership from more school administrators as 
research shows principals are pivotal in implementing innovations. 

• Teachers and principals need to develop time and arrangements for teacher 
collaboration on the standards-based mathematics curriculum and instruction 
while accommodating daily campus functioning. In addition, schools need to 
continue to develop methods to promote more parent involvement in their 
children’s mathematics education, especially in special education classes. 

• A particular challenge to the success of the ACME project is the high stakes 
state and district assessment system. 

• A final challenge to the district is to support structures that will sustain the 
high quality professional development after the ACME project is complete. 
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Recommendations 

The strengths and challenges of the ACME project in its first year derived from the 
findings of this evaluation suggest the following actions: 

• Foster in the district’s teachers, principals, central office administrators, and 
community an understanding of the difference between superficial 
implementation and standards-based instruction that enhances children’s 
mathematical competence. As with the ACME project’s development of a 
peer coaching system, the focus should be on identifying teachers and 
principals who have an understanding of the complexity of standards-based 
mathematics education and implications for practice and on developing 
networks for one-on-one or team teaching and learning. Some campuses have 
teachers and principals who are already fulfilling or could adopt this role, 
including formal leaders (i.e., curriculum specialists) and grass roots leaders 
such as teachers who have become inspired by standards-based curriculum and 
instruction. To reach every campus and teacher in the district, this effort may 
involve one hundred staff members. 

• Continue to address teachers’ various attitudes toward implementation through 
the Concerns-Based Adoption Model (CBAM; Loucks-Horsley & 
Stiegelbauer, 1992). This research-based model approaches the adoption of an 
innovation as an individualized processes and presents change as possible in a 
context that is responsive to individuals’ changing concerns. 

• Continue to provide teachers and principals with high quality professional 
development that addresses standards-based pedagogy, mathematics content, 
and instructional strategies. Continue to support the professional community 
of learners that is emerging in professional development and on campuses. 

• Encourage ACME staff and teacher leaders to improve their skills as 
facilitators by developing and carrying out a professional development plan and 
attending conferences and workshops on adult learning and leadership. 

• Continue to communicate monitor the importance of full participation of 
teachers and administrators in ACME professional development. 

• Every campus should make use of time set aside for teachers to collaborate on 
mathematics education. During professional development and other meetings, 
teachers and principals could share innovative collaborative practices across 
campuses. 

• The ACME staff need to continue public relations to address the concerns of 
principals and teachers about the relationship between standards-based 
mathematics education and the district and state assessment system. 

• Lessons learned from the ACME project’s extensive professional development 
system could be exchanged with the district’s department of professional 
development to promote and develop a powerful, self-sustaining program. 






47 



41 



97.18 



ACME. 1997-98 Report 



References 



Baretta-Lorton, M. (1978). Math their way. Cambridge, MA: Addison-Wesley. 

Batchelder, M. L. (1998, June). The Austin Collaborative for Mathematics 
Education: A Survey of Teachers Who Did Not Attend Professional Development , 1997- 
1998. (AISD, Publication No. 96-D). Austin, TX: Austin Independent School District. 

Cohen, D. K. (1990). A revolution in one classroom: The case of Mrs. Oublier. 
Educational Evaluation and Policy Analysis, 12(3), 327-346. 

Cohen, D. K., & Ball, D. L. (1990). Policy and practice: An overview. 
Educational Evaluation and Policy Analysis, 12(3), 347-353. 

Horizon Research, Inc. (1998, February). 1997-1998 Local Systemic Change: 
Revised classroom observation protocol. Chapel Hill, NC: Author. 

Horizon Research, Inc. (1998, February). 1997-1998 Local Systemic Change: 
Revised professional development observation protocol. Chapel Hill, NC: Author. 

Loucks-Horsley, S., & Stiegelbauer, S. (1992). Using knowledge of change to 
guide staff development. In A. Lieberman & L. Miller (Eds ), Staff development for 
Education in the ‘90s: New demands, new realities, new perspectives. NY: Teachers 
College Columbia University. 

National Council of Teachers of Mathematics (1989). Curriculum and Evaluation 
Standards for School Mathematics. Reston, V A: NCTM. 

National Council of Teachers of Mathematics (1991). Professional Standards for 
Teaching Mathematics. Reston, V A: NCTM. 

National Council of Teachers of Mathematics (1995). Assessment Standards for 
School Mathematics. Reston, VA: NCTM. 

Russell, S. J. (1998). Mathematics curriculum implementation: Not a beginning, 
not an end. Hands On! Hands On Math and Science Learning, 21(1), 6-9, 29. 

Weiss, I. R., Rapp, K. A., & Montgomery, D. L. (1997, October). Highlights of 
the Local Systemic Change Through Teacher Enhancement: Year Two Cross-Site Report. 
Chapel Hill, NC: Horizon Research, Inc. 

Williams, H. (1998, July). Texas Assessment of Academic Skills: 1997-98 Report. 
(AISD, Publication No. 97-12). Austin, TX: Austin Independent School District. 



O 

ERIC 



42 



48 



97.18 



ACME. 1997-98 Report 



APPENDICES 




43 



49 



Form Approval OMB No: 3145-0161 Expires: Sept. 30, 1998 




DesJgnExpcrt™ by NC5 Printed in U.S.A. Mark Reflex® EW-21 0893-1 :654321 HR06 



PLEASE DO NOT WRITE IN TIES AREA 



oooooooooooooooooooooooo 



o 






ERIC 



son Research, Inc. 



National Science Foundation 



BEST COPY AVAILABLE 



The National Science Foundation's Local Systemic Change (LSC) through Teacher 

Enhancement Program's Core Evaluation 



You have been selected to participate in the nationwide evaluation of the federally-funded Local Systemic 
Change (LSC) program. LSC is a National Science Foundation Teacher Enhancement program that is 
currently funding over 50 local projects that offer science and mathematics professional development to 
teachers in 24 states around the country. The cover letter accompanying this questionnaire identifies the LSC 
project in your area. 

A variety of strategies 

The general purpose of LSC projects is to offer teachers high-quality professional development in content 
and pedagogy. These activities are based on the national standards for reforming science and mathematics 
education. LSC projects are reaching teachers in grades K-12, although most local projects focus on either 
elementary or secondary teachers. LSC initiatives are helping teachers around the country to implement 
quality science and mathematics curriculum materials. The size, strategies, and activities of the individual 
LSC projects vary widely based on local needs. 

The national evaluation 

The National Science Foundation is accountable to Congress for the programs it funds, and the purpose of 
the LSC core evaluation is to provide both the leadership at NSF, and ultimately Congress, with information 
about the quality and impact of the Local Systemic Change program. This national evaluation is a system foi 
collecting similar information from all LSC projects through various means, including teacher and principal 
questionnaires. A small number of randomly-selected teachers in each project is asked to provide additional 
information in interviews, sometimes in conjunction with a classroom visit. In order to continue receiving 
federal funding, each LSC project must participate in this national evaluation. 

This questionnaire 

Each LSC project will administer questionnaires each spring to a randomly-selected sample of teachers who 
are targeted to participate in the local project's professional development activities. (A different group of 
teachers will be selected each year, but there is a chance over the course of several years that you could be 
selected to participate again in the future. For statistical reasons, some smaller LSC projects must administer 
this questionnaire to each participating teacher annually.) Note that you may be asked to complete this 
questionnaire even if you have not yet participated in the project's professional development; your response i: 
important, regardless of whether you have already participated. 

Confidentiality 

Data collection procedures have been developed to ensure high quality data and protect teacher 
confidentiality. Your responses will be kept strictly confidential; they will be combined with the responses 
of the other teachers in your project and used only for the LSC evaluation. The name label and numbering or 
this questionnaire are used to help local projects deliver questionnaires to the proper teachers and follow up 
with teachers that have not responded; no information identifying individual teachers will be reported under 
any circumstances. After you complete the questionnaire, you should remove the name label and return the 
questionnaire as specified by your local LSC project. 



Thank you very much for participating in this survey! 
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Instructions: Please use a #2 pencil to complete this questionnaire. Darken ovals completely, 
but do not stray into adjacent ovals. Be sure to erase completely any stray marks. 



A, 



1. 



2 . 



b. 



d. 

e. 

f. 

g- 

h. 

i. 

j- 

k. 



Teacher Opinions and Preparedness 

Please provide your opinion about each of the following statements. 
(Darken one oval on each line.) 



Strongly Disagree 
Disagree 
No Opinion 
Agree 

Strongly Agree 



a. Students generally learn mathematics best in classes with students of similar abilities. 

b. I feel supported by colleagues to try out new ideas in teaching mathematics. 

c. Teachers in this school have a shared vision of effective mathematics instruction. 

d. Teachers in this school regularly share ideas and materials related to mathematics. 

e. Teachers in this school are well- supplied with materials for investigative mathematics 
instruction. 

f. I have time during the regular school week to work with my peers on mathematics 
curriculum and instruction. 
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g. I have adequate access to calculators for teaching mathematics. 

h. I have adequate access to computers for teaching mathematics. 

i. I enjoy teaching mathematics. 

j. I am well-informed about the NCTM Standards for the grades I teach. 

k. The mathematics program in this school is strongly supported by local organizations, 
institutions, and/or businesses. 
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In the left section, please rate each of the following in terms of its importance for effective mathematics 
instruction in the grades you teach. In the right section, please indicate how prepared you feel to do each one. 
(Darken one oval in each section on each line.) 



Importance 





Not 


Somewhat 


Fairly 


Important 


Important 


Important 


Provide concrete experience before 
abstract concepts. 
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Develop students' conceptual 
understanding of mathematics. 


© 


© 


© 


Take students' prior understanding 
into account when planning 
curriculum and instruction. 
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Practice computational skills and 
algorithms. 


© 


© 
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Make connections between 
mathematics and other disciplines. 


© 


© 
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Have students work in cooperative 
learning groups. 


© 


© 


© 


Have students participate in 
appropriate hands-on activities. 


© 


© 
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Engage students in inquiry-oriented 
activities. 


© 


© 


© 


Use calculators. 


© 


© 


© 


Use computers. 


© 
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Engage students in applications of 
mathematics in a variety of contexts. 


© 


© 


© 



Preparation 



Very 

Important 


Not 

Adequately 

Prepared 


Somewhat 

Prepared 


Fairly 

Well 

Prepared 


Very 

Well 

Prepared 


© 


CD 


© 


© 
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(continued) Importance 





Not 


Somewhat 


Fairly 




Important 


Important 


Important 


Use performance-based assessment. 


© 


CD 


CD 


Use portfolios. 


© 


CD 


CD 


Use informal questioning to assess 
student understanding. 


© 


CD 


CD 



Preparation 



Very 

Important 


Not 

Adequately 

Prepared 


Somewhat 

Prepared 


Fairly 

Well 

Prepared 


Very 

Well 

Prepared 


• © 


© 


© 


© 


© 


CD 


© 


© 


© 


© 


CD 


© 


© 


© 


© 



My school principal: (Darken one oval on each line.) 



Strongly Disagree 
Disagree 
No Opinion 
Agree 

Strongly Agree 



a. Encourages me to select mathematics content and instructional strategies that address 
individual students' learning. 

b. Accepts the noise that comes with an active classroom. 

c. Encourages the implementation of current national standards in mathematics education. 

d. Encourages innovative instructional practices. 

e. Enhances the mathematics program by providing me with needed materials and equipment. 

f. Provides time for teachers to meet and share ideas with one another. 

g. Encourages me to observe exemplary mathematics teachers. 

h. Encourages teachers to make connections across disciplines. 

i. Acts as a buffer between teachers and external pressures (e.g., parents). 



© © © © © 
CD CD W CD CD 
CD CD CD CD CD 
CD CD CD CD CD 
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CD CD CD CD CD 
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Many teachers feel better prepared to teach some subject areas than others. How well prepared do you feel to teach each 
of the following subjects at the grade levels you teach, whether or not they are currently included in your curriculum? 
(Darken one oval on each tine.) 







Not 




Fairly 


Very 






Adequately 


Somewhat 


Well 


Well 






Prepared 


Prepared 


Prepared 


Prepared 


a. 


Science 


© 


© 


© 


© 


b. 


Mathematics 


©: 


© 


© 


© 


c. 


Reading/Language Arts 


© 


© 


CD 


© 


d. 


Social Studies 


©: 


© 


© 


© 



Within mathematics, many teachers feel better prepared to teach some topics than others. How well prepared do you feel to 
teach each of the following topics at the grade levels you teach, whether or not they are currently included in your 
curriculum? (Darken one oval on each line.) 







Not 




Fairly 


Very 






Adequately 


Somewhat 


Well 


Well 






Prepared 


Prepared 


Prepared 


Prepared 


a. 


Numeration and number theory 


© • 


© 


© 


© 


b. 


Computation 


© 


CD 


© 


© 


c. 


Estimation 


© 


CD 


© 


© 


d. 


Measurement 


© 


CD 


© 


© 


e. 


Pre-algebra 


© 


CD 


© 


© 


f. 


Algebra 


CD 


CD 


© 


© 


g- 


Patterns and relationships 


© 


CD 


© 


© 


h. 


Geometry and spatial sense 


© 


CD 


© 


© 


i. 


Data collection and analysis 


© 


CD 


© 


© 


j- 


Probability 


© 


CD 


© 


© 


k. 


Technology (calculators, computers) in support of mathematics 


© 


CD 


© 


© 



Horizon Research, Inc. 




Spring 199 



2 



6. Please indicate how well prepared you feel to do each of 



the following. (Darken one oval on each line.) 




Not 






Fairly 




Very 




Adequately 


Somewhat 




Well 




Well 






Prepared 


Prepared 




Prepared 


Prepared 


a. Lead a class of students using investigative strategies. 




CD 


© 




© 




© 


b. Manage a class of students engaged in hands-on/project-based 


CD 


© 




© 




© 


work. 




<D 


© 




© 




© 


c. Help students take responsibility for their own learning. 




CD 


© 




© 




© 


d. Recognize and respond to student diversity. 

e. Encourage students' interest in mathematics. 




CD 


© 




© 




© 


f. Use strategies that specifically encourage participation of females 


CD 


© 




© 




© 


and minorities in mathematics. 




CD 


© 




© 




© 


g. Involve parents in the mathematics education of their students. 














Please rate the effect of each of the following on your mathematics instruction. 












(Darken one oval on each line.) 


















Inhibits 








Encourages 






effective 




Neutral 




effective 


NA / Don't 




instruction 




or mixed 




instruction 


Know 


a. State and/or district curriculum frameworks. 


CD 


CD 


© 


© 




© 


© 


b. State and/or district testing policies and practices. 


CD 


CD 


© 


© 




© 


© 


c. Quality of available instructional materials. 


CD 


CD 


© 


© 




© 


© 


d. Access to calculators for mathematics instruction. 


CD 


CD 


© 


© 




© 


© 


e. Access to computers for mathematics instruction. 


CD 


CD 


© 


© 




© 




f. Funds for purchasing equipment and supplies for 
















mathematics. 


CD 


CD 


© 


© 




©: 


© 


g. System of managing instructional resources at the district 














or school level. 


CD 


CD 


© 


© 




© 


© 


h. Time available for teachers to plan and prepare lessons. 


CD 


© 


© 


© 




© 


© 


i. Opportunities for teachers to work with other teachers. 


CD 


© 


© 


© 




© 


© 


j. Opportunities for teacher professional development. 


CD 


© 


© 


© 




© 


© 


k. Importance that the school places on mathematics. 

l. Consistency of mathematics reform efforts with other 


CD 


© 


© 


© 




© 


© 


school/district reforms. 


CD 


© 


© 


© 




© 


© 


m. Public attitudes toward reform. 


CD 


© 


© 


© 




© 


© 


How many of your students' parents do each of the following? 














(Darken one oval on each line.) 






Few or 




About 




Almost 








None 




1/2 




All 


a. Volunteer to assist with class activities. 






© 


© 


© 


© 


© 


b. Donate money or materials for classroom instruction. 






© 


© 


© 


© 


© 


c. Attend parent- teacher conferences. 






© 


© 


.© 


© 


© 


d. Attend school activities such as PTA meetings and Family Mathematics nights. 


© 


© 


© 


© 


© 


e. Voice support for the use of an investigative approach to mathematics instruction. © 


© 


© 


© 


© 


f. Voice support for traditional approaches to mathematics instruction. 




© 


© 


© 


© 


© 



B. Your Mathematics Teaching 

9. What grade level (s) are you currently teaching? 
(Darken all ovals that apply.) 
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2 
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3 4 5 6 7 8 9 10 11 12 
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Questions 10-12d ask about your mathematics teaching. If you teach more than one mathematics class, 
please answer for your first class of the day. 

10 . 



11 . 



HD 
HD 
HD 
HD 
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HD 
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About how often do you do each of the following in your mathematics 




Rarely 


Sometimes 


Often 


All or 


instruction? (Darken one oval on each line.) 




(e.g., a few 


(e.g., once 


(e.g., once 


almost all 




vr 


times a 


or twice 


or twice 


mathematics 






year) 


a month) 


a week) 


lessons 


a. 


Introduce content through formal presentations. 


CD 


CD 


CD 


CD 


© 


b. 


Arrange seating to facilitate student discussion. 


CD 


CD 


CD 


© 


© 


c. 


Use open-ended questions. 


CD 


CD 


CD 


© 


© 


d. 


Require students to explain their reasoning when giving an 














answer. 


CD 


CD 


CD 


© 


© 


e. 


Encourage students to communicate mathematically. 


CD 


CD 


CD 


© 


© 


f. 


Encourage students to explore alternative methods for 














solutions. 


CD 


CD 


CD 


© 


© 


g- 


Encourage students to use multiple representations (e.g., 














numeric, graphic, geometric, etc.). 


CD 


CD 


CD 


© 


© 


h. 


Allow students to work at their own pace. 


© 


CD 


CD 


© 


© 


i. 


Use assessment to find out what students know before or 














during a unit. 


CD 


CD 


CD 


© 


;© 


j- 


Embed assessment in regular class activities. 


CD 


CD 


CD 


© 


;© 


k. 


Assign mathematics homework. 


© 


CD 


CD 


© 




1. 


Read and comment on the reflections students have written in 










;A- 




their notebooks or journals. 


CD 


CD 


CD 


© 


© 


About how often do students in this class take part in each of 




Rarelv 


Sometimes 


Often 


All or 


the following types of activities as part of their mathematics 




(e.g., a few (e.g., once 


(e.g., once 


s almost all 


instruction? (Darken one oval on each line.) 


NT 

Never 


times a 


or twice 


or twice 


mathematic: 








year) 


a month) 


a week) 


lessons 


a. 


Participate in student-led discussions. 


CD 


CD 


CD 


© . 


;© 


b. 


Participate in discussions with the teacher to further 














mathematical understanding. 


CD 


CD 


CD 


© 


© 


c. 


Work in cooperative learning groups. 


CD 


CD 


CD 


© 


© 


d. 


Make formal presentations to the class. 


CD 


CD 


CD 


© 


© 


e. 


Read from a mathematics textbook in class. 


CD 


CD 


CD 


© 


■S; 


f. 


Read other (non-textbook) mathematics-related materials in 


CD' 


CD 


CD 


© 


© 




class. 


m 


CD 


CD 


© 


CD 


g- 


Practice routine computations/algorithms. 


CD 


CD 


CD 


© 


:©. 


h. 


Review homework/worksheet assignments. 


CD 


CD 


CD 


© 


© 


i. 


Use mathematical concepts to interpret and solve word problems. 


© 


CD 


CD 


© 


CD 


j- 


Work on solving a real-world problem. 


:© 


CD 


CD 


© 


© 


k. 


Share ideas or solve problems with each other in small groups. 
















CD 


CD 


CD 


© 


©: 


1. 


Engage in hands-on mathematical activities. 


CD 


CD 


CD 


© 


© 


m. 


Play mathematics games. 


CD 


CD 


CD 


© 


© 


n. 


Follow specific instructions in an activity or investigation. 


CD 


CD 


CD 


© 


© 


0. 


Design or implement their own investigation. 


CD 


CD 


CD 


© 


© 


P- 


Work on models or simulations. 












q- 


Work on extended mathematics investigations or projects (a 


CD 


CD 


CD- 


© 


© 




week or more in duration). 


CD 


CD 


CD 


© 


© 


r. 


Participate in field work. 


CD 


CD 


CD 


© 


© 


s. 


Record, represent and/or analyze data. 












t. 


Write a description of a plan, procedure or problem-solving 


<D 


CD 


CD 


© 


© 




process. 


CD 


CD 


© 


© 


© 


u. 


Write reflections in a notebook or journal. 
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Never 


Rarely 
(e.g., a few 
times a 
year) 


Sometimes 
(e.g., once 
or twice 
a month) 


Often 
(e.g., once 
or twice 
a week) 


All or 
almost all 
mathematic! 
lessons 


v. Use calculators or computers for learning or practicing skills. 


© 


CD 


CD 


CD 


CD 


w. Use calculators or computers to develop conceptual understanding. 

x. Use calculators or computers as a tool (e.g., spreadsheets, data 


■ CD 


CD 


CD 


CD 


CD 


analysis). 


CD 


CD 


CD 


CD 


CD 


y. Work on portfolios. 


CD 


CD 


CD 


CD 


CD 


z. Take short-answer tests (e.g., multiple choice, true/false. 


fill -in -the -blank). 

aa. Take tests requiring open-ended responses (e.g., descriptions. 


© 


CD 


CD 


CD 


CD 


justifications of solutions). 


CD 


CD 


CD 


CD 


CD 


bb. Engage in performance tasks for assessment purposes. 


CD 


CD 


CD 


CD 


CD 



12a. Do you teach in a self-contained classroom? 

O Yes O No (Skip to Question 13) 

12b. How many lessons per week do you typically teach mathematics to this class? 

Number of Lessons 

0 1 2 3 4 5 

o o o o o o 

12c. Approximately how many minutes is a typical mathematics lesson? 

Average Number of Minutes per Lesson 



<10 


11-20 


21-30 


31-40 


41-50 


51-60 


61-70 


71-80 


81 or more 


o 


O 


O 


O 


O 


o 


O 


O 


O 



In how many of the last five school days did you teach each 
of the following in this class? (Darken one oval on each line.) 






Number 


of Days 








none 


one 


two 


three 


four 


five 


a. Science 


O 


O 


O 


O 


O 


O 


b. Mathematics 


O 


o 


O 


O 


O 


o 


c. Reading/Language Arts 


o 


o 


o 


O 


o 


o 


d. Social Studies 


o 


o 


o 


O 


o 


o 



C. LSC Professional Development 

Questions 13-17 refer to the NSF-supported Local Systemic Change (LSC) program. Please refer to the letter accompanying 
this questionnaire for information about the LSC project activities in your district. If you have not yet participated in LSC 
professional development^darken this oval O and skip to Question 17 . 



13. 



To what extent is each of the following true of LSC mathematics-related professional development in your district? 
(Darken one oval on each line.) 



a. I am involved in planning my mathematics-related professional development. 

b. I am encouraged to develop an individual professional development plan to address my 
needs and interests related to mathematics education. 

c. I am given time to work with other teachers as part of my professional development. 

d. I am given time to reflect on what I've learned and how to apply it to the classroom. 

e. I receive support as I try to implement what I've learned. 



Not To a great 

at all extent 



© 


© 


© 


© 


© 


CD 


© 


© 


© 


© 


CD 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 


© 
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Approximately how many hours have you spent on 15. How would you rate the overall quality of the L SC 

professional development in mathematics/mathematics professional development? (Darken one oval.) 

education as part of the LSC project? (Darken one oval.) 



o 


0 


O 


80-99 


O 


Very Poor 


o 


1-9 


o 


100-129 


O 


Poor 


o 


10-19 


o 


130-159 


O 


Fair 


o 


20-39 


o 


160-199 


O 


Good 


o 


40-59 


o 


200 or greater 


O 


Very Good 


o 


60-79 






. o 


Excellent 



To what extent has participation in LSC mathematics-related professional development increased your: 

To a 

Not at all great extent 



a. 


mathematics content knowledge 


CD 


CD 


CD: 


G> 


CD 


b. 


understanding of how children think about/leam mathematics 


CD 


CD 


CD 


CD 


CD 


c. 


ability to implement high-quality mathematics instructional materials 


CD 


<D 


<D : 


CD 


CD 



Have you been identified as a lead teacher for 
your district’s NSF-sponsored LSC project? 

® Yes 
O: No 



Teacher Demographic Information 



18. 


Are you: 


19. Are 


you: 




Q 


Male 


O 


African-American (not of Hispanic origin) 






Female 


Q 


American Indian or Alaskan Native 








o 


Asian or Pacific Islander 








o 


Hispanic 


20. 


How many college mathematics courses have you 


o 


White (not of Hispanic origin) 




completed? (Darken one Oval.) 


Q; 


Other 




:Q : 


none 








W'- 


1 semester 


21. Did 


your college mathematics coursework 




M 


2 semesters 


include the equivalent of at least one 




m 


3 semesters 


semester of: 




o 


4 semesters 


(Darken one oval on each line.) Yes No 




o 


5 or more semesters 


a. 


number system concepts Q O 








b. 


concepts in algebra £3 O 


22. 


How 


many years have you taught prior to this school year? 


c. 


concepts in geometry O O 



(Darken one oval.) 



0-2 


3-5 


6-10 


11-20 


21 or more 


OFF'C 




•/ ° 




WM' 
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1998 Local Systemic Change 
Principal Questionnaire 



Form Approval 
OMB No: 3145-0161 
Expires: Sept. 30, 1998 



Instructions: Please use a #2 pencil to complete this questionnaire. Darken ovals completely, but do not 
stray into adjacent ovals. Be sure to erase completely any stray marks. 



A. Instruction 



1. Please indicate the extent to which your school is involved in reform in each of the following areas. 
(Darken one oval on each line.) 





No reforms 


Exploring 


Initial stages 


Heavily involved 




being discussed 


possibilities 


of reform 


in reform 


a. Language Arts 


© 


CD 


CD, 


CD 


b. Mathematics 


© 


CD 


© 


CD 


c. Science 


© 


© 


© 


CD 


d. Social Studies/History 


© 


© 


© 


CD 


e. Technology 


CD 


© 


CD 


CD 


f. Integrating the curriculum across disciplines 


•CD. 


© 


CD 


CD 


g. Site-based management 


CD 


© 


<p! 


CD 



2 . 



Please provide your opinion about each of the following statements regarding science and mathematics instruction. 
(Darken one oval in each section on each line.) 



Science 



Mathematics 





Strongly 




No 




Strongly 


Strongly 




No 




Strongly 


a. Students generally learn best in classes 


disagree 




opinion 




agree 


disagree 




opinion 




agree 


with students of similar abilities. 


jct>; 


© 


© 


© 




;©;. 


© 


© 


© 


W i 


b. I am knowledgeable about the current national 






















standards in this content area. 


© 


© 


CD 


© 


m 


:© 


© 


CD: 


© 


m 


c. I feel well-prepared to support teachers in the 






i? 










v,- ■ 




\ .V 
•** • ■ 


implementation of current national standards. 


© 


© 


© 


© 


m 


©; 


© 


:©; 


© 


© 


d. I am willing to accept the noise that comes with 






















an active classroom. 


ph 


© 


© 


© 


:©! 


m 


© 


©: 


© 


© 


e. Encouraging student questions is more important 




















k, ■ ; 


than eliciting correct answers. 


© 


© 


© 


© 


CD 




© 


:©: 


© 


w 



3 . 



Please rate each of the following in terms of its importance for effective science and mathematics instruction. 

(Darken one oval in each section on each line.) 

Science Mathematics 





Not 


Somewhat 


Fairly 


Very 


Not 


Somewhat 


Fairly 


Veiy 


a. Provide concrete experience before abstract 


Important Important Important Important 


Important Important Important Important 


concepts. 

b. Develop students' conceptual understanding of 


CD 


© 




© 




© 


}©■: 


© 


the subject. 

c. Take students' prior understanding of subject 
matter into account when planning curriculum 


CD 


© 


!©; 


© 


© 


© 


| © 


© 


and instruction. 


i© 


© 


© 


© 


CD 


© 


:© 


© 


d. Practice computational skills and algorithms. 


:© 


© 


;©: 


© 


1©: 


© 


Q> 


© 


e. Make connections to other disciplines. 


CD 


© 


© : 


© 


CD 


© 


© 


© 



PLEASE DO NOT: WRITE IN THIS AREA 
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(continued) 


















feo 1 


Science 










Mathematics 




f59l 




















1 58 | 


Not Somewhat 


Fairly 


Very 






Not Somewhat Fairly 


Very 


y 


f. Have students work in cooperative learning Important Important Important Important 




Important Important Important Importan 


1 56 1 


groups. CD CD 


CD 


CD 






0 


CD 


0 


© 


HU 


g. Have students participate in appropriate 


















mi 


hands-on activities. CD CD 


CD 


CD 






CD 


CD 


CD 


© 


im 


h. Engage students in inquiry -oriented activities. CD CD 


<D 


CD 






0 


CD 


0; 


© 


1 52 | 


i. Use calculators. CD CD 


CD 


CD 






CD 


CD 


CD 


© 


dD 


j. Use computers. CD CD 


m 


CD 






0 


. CD 


0 


© 


do] 




















HD 


k. Engage students in applications of subject Jd 


















HD 


matter in a variety of contexts. CD CD 


© 


CD 






01 


CD 


:0I 


© 


HD 


1. Use performance-based assessment. CD CD 


0. 


CD 






0 


CD 


0: 


© 


1 46 | 


m. Use portfolios. [0; CD 


CD 


CD 






0: 


CD 


0 


© 


1 45 | 


n. Use informal questioning to assess student i 


















1 44 | 


understanding. CD CD 


CD 


CD 






m 


CD 


0 


© 


hd 




















1 42 | 




















1 41 | 




















1 40 | 




















1 39 1 




















[~38~1 


4. Please rate the effect of each of the following on science instruction in your 
















dz] 


school. (Darken one oval on each line.) 


















del 


Inhibits 












Encourages 


NA/ 


dD 


effective 






Neutral 






effective 


Don’t 


HD 


instruction 






or mixed 




instruction 


Know 


dE 


a. State and/or district curriculum frameworks. 


CD 




0 




CD 




' 0; 


© 


□D 


b. State and/or district testing policies and practices. 


CD 




g| 




CD 




iH 


© 


dD 


c. State, district, and/or school grading policies and 


















do] 


practices. (CP 


CD 




0 : 


CD 




CD 


© 


dE 


d. Counseling department policies and practices. CD 


CD 




CD 




CD 




0 


© 


Ids] 


e. College placement tests. 0J 


CD 




0 




CD 




M 


© 


dD 


i'kkl 






p 












dE 


f. District/school structures for recognizing and ^ 






SB 












dD 


rewarding teachers. 0 


CD 




0 




CD 




§g 


© 


dD 


g. Quality of available instructional materials. pj 


CD 




0 




CD 




0 


© 


dD 


h. Access to computers for science instruction. 0 


CD 




0 




CD 




0| 


© 


dD 


i. Funds for purchasing equipment and supplies for jy 






[••D. 












dD 


science. 0 


CD 




0 




CD 




0| 


© 


Idol 


j. System of managing instructional resources at the 


















dD 


district or school level. CD 


CD 




0 




CD 




0 


© 


dD 


k. Time available for teachers to plan and prepare lessons. CD- 


CD 




0 




CD 




01 


© 


dD 








‘ 












dD 


1. Opportunities for teachers to work with other teachers. ;0 


CD 




10 




CD 




0j 


© 


dD 


m. Opportunities for teacher professional development. 0 


CD 




CD 




0 




m 


© 


dD 


n. Importance that the school places on science. 0 


CD 




0 




CD 




CD 


© 


dD 


0 . Consistency of science reform efforts with other ivip 


















dD 


school/district reforms. j 0 


CD 




CD 




CD 




0j 


© 


dD 


p. Public attitudes toward reform. ;0 


CD 




0 




CD 




0: 


© 


dD 




















ra 




















ra 




















ra 




















ra 




















m 




















ra 




















rao 
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Please rate the effect of each of the following on mathematics instruction in your school. 
(Darken one oval on each line.) 





Inhibits 








Encourages 




effective 




Neutral 




effective 


NA/ Don’t 


instruction 




or mixed 




instruction 


Know 


a. State and/or district curriculum frameworks. 


:© 


CD 


CD 


CD 


CD 


. © 


b. State and/or district testing policies and practices. 


CD 


CD 


CD 


© 


CD 


© 


c. State, district and/or school grading policies and practices 


. CD 


CD 


CD 


© 


CD 


© 


d. Counseling department policies and practices. 


CD 


CD 


CD 


CD 


(D 


© 


e. College placement tests. 


CD 


CD 


CD 


CD 


CD 


© 


f. District/school structures for recognizing and rewarding 














teachers. 


CD 


CD 


CD 


CD 


CD 


© 


g. Quality of available instructional materials. 


•CD 


CD 


CD 


CD 


;<D: 


© 


h. Access to calculators for mathematics instruction. 


CD 


CD 


CD 


CD 


;<D 


© 


i. Access to computers for mathematics instruction. 


CD 


CD 


CD 


CD 


m- 


© 


j. Funds for purchasing equipment and supplies for 






fY! 








mathematics. 


CD 


CD 


;CD 


CD 


CD 


© 


k. System of managing instructional resources at the district 










If' 




or school level. 


CD 


CD 


CD 


CD 


■g>] 


© 


1. Time available for teachers to plan and prepare lessons. 


CD 


CD 


CD: 


CD 


CD 


© 


m. Opportunities for teachers to work with other teachers. 


CD 


CD 


CD 


CD 


CD 


© 


n. Opportunities for teacher professional development. 


m. 


CD 


CD 


CD 


;© 


© 


o. Importance that the school places on mathematics. 


CD 


CD 


CD 


CD 


;©: 


© 


p. Consistency of mathematics reform efforts with other 






v ‘-> 








school/district reforms. 


CD 


CD 


CD 


CD 


CD 


© 


q. Public attitudes toward reform. 


j<D 


CD 


CD 


CD 


CD 


© 



To what extent: 



(Darken one oval on each line.) 

Not at all 

a. Are you familiar with the LSC project in your district? CD 


(D 


CD 


CD 


To a 

great extent 

[CD' 


Don't 

know 


b. 


Have you been involved in LSC project activities? 


CD 


CD 


CD 


CD 


[© 


© 


c. 


Have teachers in your school been involved in LSC project activities? 


;CD; 


CD 


;CD' 


CD 


CD; 


© 


d. 


Are teachers in your school using the LSC-specified instructional 
materials with their students? 




CD 


LCD' 


© 


:©j 


© 


e. 


Are teachers in your school using the LSC-advocated instructional 
strategies with their students? 


M 


CD 


© 


CD 


m 


© 



How would you describe your school's progress in moving toward excellence in science and mathematics education? 
(Darken one oval on each line.) 



Quite far 




Beginning to 


Well along in 


Approaching 








from ideal 




improve 


improving 


ideal 








a. Science program CD 


© 


CD CD 


;<d © 


© 








b. Mathematics program ( Dj 


© 


CD CD 


m cd 










Principal Information 
















8. Including this year, how many years have you been: 












(Darken one oval on each line.) 














More 






1 


2 3 


4 5 


6-10 


11-20 


than 20 


a. A principal? 




CD 


© ;© 


CD CD 


CD 


© 


® 


b. The principal at this school? 




© 


CD CD 


CD -CD 


(D 


© 


CD 


c. A principal in this school district? 




© 


© D 


CD CD 


CD 


©: 


CD 



m Research, Inc. 



3 



Spring 1998 



[63] 

[62] 

HE 

m 

HE C. 

HE 

HE 

HE 9. 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE i2. 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 



HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

HE 

m 

m 

m 

m 

ca 

m 

IERLC 



School Characteristics 



-Place Label Here ~ 



How many 
students attend 
your school? 
(Please enter your 
response as a four 
digit number and 
then darken the 
appropriate oval in 
each column. For 
example, enter 
850 students as 
0850.) 



000 CD 
CD CD CD CD 
CD CD CD CD 
CD CD CD CD 
CD CD CD CD 
CD CD CD CD 
0 00 0 
CD CD fD CD 
CD-CD;Cp CD 
0 0:0 0 



students 



10. In what type of community is this school located? (Darken one oval 
CD Rural 

O Town or Small City 
CD Suburban 
O Urban 

1 1 . This school includes the following grades: (Darken all that apply.) 



K 


1 


2 


3 


4 


5 6 


qg 


gO; :: 


:'<P' 




;o| 


° i§3 


7 


8 


9 


10 


11 


12 


111 


O 


fo | 


0 




i||; 



Approximately what percentage of the students attending this school are: (Please enter each as a three-digit 
number and then darken the appropriate oval in each column. For example, enter 25 percent as 025, enter 
less than 1/2 percent as 000.) 



a. American Indian 
or Alaskan Native? 



b. African- 
American? 



c. Asian or 
Pacific Islander? 



d. Hispanic, 
regardless of race? 



e. White (not of 
Hispanic origin)? 









CD 


® 


CD 


CD 


;© 


O 




CD 


CD 




CD 


CD 




j® 


CD 




CD 


CD 




CD 


CD 




CD 


CD 




CD 


CD 




jcD 


CD 



% 



CD CD CD 
CD CD CD 
5 CD I C§ 
cd;cd 

CD CD 
CD CD 
;CD; CD 
CD CD 
CD CD 
CD CD 



% 



CD CD CD 
CD ; CD CD 
CD CD 
I; CD CD 
wfiC D CD 
• CD CD 
CD CD 
|| CD CD 

J|c d 0 

... 0;® 



% 



0 0® 
CD®© 
;® CD 
;C D© 
CD CD 
i® ® 
0 0 
CD CD 
; ® CD 
| CD CD 



% 



CD CD CD 
©®® 
:®<D 
CD CD 

|®® 
| CD CD 
i|| CD® 
| CD CD 
CD CD 
CD -CD 



% 



13. 



What is the 








14. What percentage of 
























estimated 








% the students in this 








% 




For office use only 


percentage of 


CD CD CD 


school are eligible 


CD CD ® 




. 














students in this 


CD 0:0 


for free or reduced 


CD CD CD 




l 














school with limited 


;..y 


CD CD 


price lunches that 




CD CD 








CD 


CD 


CD 


0 


0 


English 


|| 


CD (D 


are paid for with 




CD CD 








CD 


0 


CD 


0 


0 


proficiency? 




CD CD 


public funds? 




CD 0 




1 




CD 


CD 


CD 


CD 


0 






CD CD 






CD CD 




■: yMff: 




CD 


0 


CD 


0 


0 






CD;<D 




• 40® 








CD 


0 


CD 


0 


0 






CD CD 






C Dp 




y: : •• 


CD 


CD 


0 


CD 


0 


0 






CD CD 






CD CD 




n' *.:< 




CD 


0 


CD 


CD 


0 






CD CD 






CD 


i> 




' Ev',::, M 




CD 


CD 


CD 


0 


0 


















f 




CD 


0 


0 


0 


0 




















CD 


CD 


CD 


0 


0 


0 
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1997-98 Local Systemic Change 
Pre-Classroom Observation Interview 



After you have expressed appreciation to the teacher for allowing you to observe the 
class, ask the following question: 

1. What has this class been doing in mathematics/science recently? 

PROBES: What unit are you working on? 

What instructional materials are you using 1 ? 



2. What do you anticipate doing in your mathematics/science class on the day I will 
be observing? 

PROBE: What do you hope students will learn as a result of the work you 

have planned? 

3. What is the next step for this class? 

4. Is there anything in particular that I should know about the group of students that 
I will be observing? 



Note that the evaluator will need to be thoroughly conversant with the instructional materials designated for use by the LSC in order 
to complete the observation ratings. 
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NOTE 



This form is included for information purposes only. Evaluators will need to 
complete the form on the Web or request scannable forms from HRI. 



1997-98 Local Systemic Change 
REVISED Classroom Observation Protocol 1 



Background Information 



Project Date of Observation 

LSC ED 2 Time of Observation: 



Random Sample Backup Sample 



Start 



End 



Other, specify 

Subject Observed 3 



Observer 



Grade Level 



Observer’s Role in Project: 

Project Evaluator Local Observer 

Lead 

Other 



Section One: Contextual Background and Activities 

In this section, please fill in the circles that best describe the class. For each item, be sure to fill in 
all responses that apply. 



I. Classroom Demographics 

A. What is the total number of students in 
the class at the time of the observation? 
O 15 or fewer 
O 16-20 
O 21-25 
O 26-30 
O 31 or more 



B. What is the approximate percentage of 
white (not Hispanic origin) students in 
this class? 

O 0-10 percent 
O 1 1-25 percent 
O 26-50 percent 
O 51-75 percent 
O 76-100 percent 



C. 



Indicate the teacher’s : 

1 . Gender 

O Male O Female 

2. Race/Ethnicity 

O African-American (not Hispanic origin) 
O American Indian or Alaskan Native 
O Asian or Pacific Islander 
O Hispanic 

O White (not Hispanic origin) 

O Other 



D. If applicable, indicate the teacher aide ’ s : 

1. Gender 

O Male O Female 

2. Race/Ethnicity 

O African-American (not Hispanic origin) 
O American Indian or Alaskan Native 
O Asian or Pacific Islander 
O Hispanic 

O White (not Hispanic origin) 

O Other 



1 Be sure you have read the “1997-98 Local Systemic Change Classroom Observations: Guidelines for Evaluators” and have completed the “Pre- 
Classroom Observations Interview” before observing the class. 

2 Use the LSC ID number as indicated in the Classroom Observation Sample provided by HRI. 

3 In mathematics/science projects observe the subject for which the teacher was sampled. 
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II. Classroom Context 



A. Rate the adequacy of the physical environment. 

1. Classroom resources: 



0 O O 

1 2 3 

Sparsely equipped 

2 . Classroom Space: 

0 O O 

1 2 3 

Crowded 

3 . Room arrangement: 

0 o o 

1 2 3 

Inhibited interactions 
among students 



o o 

4 5 

Rich in resources 



o o 

4 5 

Adequate space 



o o 

4 5 

Facilitated interactions 
among students 



B. In a few sentences, describe the lesson you observed. Include where this lesson fits in the overall unit 
of study. 



III. Purposes of Lesson 

A. Indicate the major 4 content areafs) of this lesson or activity. 



O 1. 


Numeration and number theory 


O 16. 


Life Science 


O 2. 


Computation 


O 17. 


Physical science 


O 3. 


Estimation 


O 18. 


Earth/space sciences 


O 4. 


Measurement 


O 


a. Astronomy 


O 5. 


Patterns and relationships 


O 


b. Oceanography 


O 6. 


Pre-algebra 


O 


c. Geology 


O 7. 


Algebra 


O 


d. Meteorology 


O 8. 


Geometry and spatial sense 


O 


e. Environmental science 


O 9. 


Functions (including trigonometric 


O 19. 


Engineering and design principles 




functions) and pre-calculus concepts 


O 20. 


History of mathematics/science 


O 10. 


Data collection and analysis 






O 11. 


Probability 


O 21. 


None of the above (please explain) 


O 12. 


Statistics (e.g., hypothesis tests, 








curve-fitting, and regression) 






O 13. 


Topics from discrete mathematics 








(e.g., combinatorics, graph theory. 








recursion) 






O 14. 


Mathematical structures (e.g., vector spaces. 








groups, rings, fields) 






O 15. 


Calculus 







“Major” means was used or addressed for a substantial portion of the lesson; if you were describing the lesson to someone, this feature would help 
characterize it. 



3 
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B. Indicate the major 5 intended purpose(s) of this lesson or activity based on the pre- and/or post- 
observation interviews with the teacher. 

O 1 . Identifying prior student knowledge 

O 2. Introducing new concepts 

O 3. Developing conceptual understanding 

O 4. Reviewing mathematics/science concepts 
O 5. Developing problem-solving skills 

O 6. Learning mathematics/science processes, algorithms, or procedures 
O 7. Learning vocabulary/specific facts 

O 8. Practicing computation for mastery 

O 9. Developing appreciation for core ideas in mathematic s/science 

O 10. Developing students’ awareness of contributions of scientists/mathematicians of diverse 
backgrounds 

O 11. Assessing student understanding 

IV. Instructional Materials 

A. Is this lesson based on commercially produced instructional materials? 

O Yes O No, SKIP to Part V below 

B. Are the instructional materials among those designated for use by this LSC? 

O Yes O No, SKIP to Part V below 

C. Indicate the single set of LSC -designated instructional materials intended to form the basis of this 
lesson (e.g., FOSS; Insights; STC; Investigations in Number, Data, and Space; Connected Math; 
IMP), based on the information provided in the pre-observation interview. 

Please specify. 



D. How closely did the lesson adhere to the instructions provided in the teacher’s manual? 

O Exactly, SKIP to Part V below 

O Almost totally O Mostly O Somewhat O A little O Hardly at all 

E. How did the adaptations affect the quality of the lesson? 

O Helped a lot O Helped a little O Neutral O Hurt a little O Hurt a lot 



V. Classroom Instruction 

A. Indicate the major 5 way(s) in which student activities were structured. 

O As a whole group O As small groups O As pairs O As individuals 

B. Indicate the majoi * way(s) in which students engaged in class activities. 

O Entire class was engaged in the same activities at the same time. 

O Groups of students were engaged in different activities at the same time (e.g., centers). 

5 “Major” means was used or addressed for a substantial portion of the lesson; if you were describing the lesson to someone, this feature would help 
characterize it. 
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C. Indicate the major activities of students in this lesson. When choosing an “umbrella” category, be sure 
to indicate subcategories that apply as well. (For example, if you mark “listened to a presentation,” 
indicate by whom.) 

O 1. Listened to a presentation: 

O a. By teacher (would include: demonstration, lectures, media presentations, extensive procedural instructions) 

O b. By student (would include informal, as well as formal, presentations of their work) 

O c. By guest speaker/“expert” serving as a resource 

O 2. Engaged in discussions/seminars: 

O a. Whole group 
O b. Small groups/pairs 

O 3. Engaged in problem solving/investigation: 

O a. Worked with manipulates 
O b. Played a game to build or review knowledge/skills 
O c. Followed specific instructions in an investigation 
O d. Had some latitude in designing an investigation 

O e. Recorded, represented and/or analyzed data 
O f. Recognized patterns, cycles or trends 
O g. Evaluated the validity of arguments or claims 
O h. Provided an informal justification or formal proof 

O 4. Engaged in reading/reflection/written communication about mathematics or science: 

O a. Read about mathematics/science 
O b. Answered textbook/worksheet questions 

O c. Reflected on readings, activities, or problems individually or in groups 
O d. Prepared a written report 

O e. Wrote a description of a plan, procedure, or problem-solving process 
O f. Wrote reflections in a notebook or journal 

O 5. Used technology/audio-visual resources: 

O a. To develop conceptual understanding 
O b. To learn or practice a skill 
O c. To collect data (e.g., probeware) 

O d. As an analytic tool (e.g., spreadsheets or data analysis) 

O e. As a presentation tool 

O f. For word processing or as a communications tool (e.g., e-mail, Internet, Web) 

O 6. Other activities 

O a. Arts and crafts activity 
O b. Listened to a story 
O c. Wrote a poem or story 

O d. Other (Please specify.) 



“Major” means was used or addressed for a substantial portion of the lesson; if you were describing the lesson to someone, this feature would help 
characterize it. 
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D. Comments 

Please provide any additional information you consider necessary to capture the activities or context of 
this lesson. Include comments on any feature of the class that is so salient that you need to get it “on the 
table” right away to help explain your ratings; for example, the class was interrupted by a fire drill, the 
kids were excited about an upcoming school event, or the teacher’s tone was so warm (or so hostile) that it 
was an overwhelmingly important feature of the lesson. 



Section Two: Ratings 

In Section One of this form, you documented what occurred in the lesson. In this section, you are 
asked to rate each of a number of key indicators in four different categories, from 1 (not at all) to 5 
(to a great extent). You may list any additional indicators you consider important in capturing the 
essence of this lesson and rate these as well. Use your “Ratings of Key Indicators” (Part A) to inform 
your “Synthesis Ratings” (Part B). It is important to indicate in “Supporting Evidence for Synthesis 
Ratings” (Part C) what factors were most influential in determining your synthesis ratings and to give 
specific examples or quotes to illustrate those factors. 

Note that any one lesson is not likely to provide evidence for every single indicator; use 6, “Don't 
know” when there is not enough evidence for you to make a judgment. Use 7, “N/A” (Not 
Applicable) when you consider the indicator inappropriate given the purpose and context of the 
lesson. Section Two concludes with ratings of the likely impact of instruction, and a capsule 
description of the lesson. 
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I. Design 

A. Ratings of Key Indicators 

1 . The design of the lesson incorporated tasks, roles, and 
interactions consistent with investigative mathematics/science. 

2. The design of the session reflected careful planning and 
organization. 

3. The instructional strategies and activities used in this 
lesson reflected attention to students’ experience, 
preparedness, and/or learning styles. 

4. The resources available in this lesson contributed to 
accomplishing the purposes of the instruction. 

5. The instructional strategies and activities reflected attention 
to issues of access, equity, and diversity for students 

(e.g., use of “wait time,” cooperative learning, language- 
appropriate strategies/materials). 

6. The design of the lesson encouraged a collaborative 
approach to learning. 

7. Adequate time and structure were provided for reflection. 

8. Adequate time and structure were provided for wrap-up 
and closure. 

9. Formal assessments of students were consistent with 
investigative mathematics/science. 

10. Design for future instruction takes into account what 
transpired in the lesson. 



11 . 



Not 

at 

all 



To a 
great 
extent 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 



1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 



Don’t 

know 



N/A 



B. Synthesis Rating 



1 


2 


3 


4 


5 


Design of the lesson not at 
all reflective of best 
practice in 
mathematics/science 
education 








Design of the lesson 
extremely reflective of 
best practice in 
mathematics/science 
education 



C. Supporting Evidence for Synthesis Rating 
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II. Implementation 



A. Ratings of Key Indicators 

1. The instruction was consistent with the underlying 
approach of the instructional materials designated 
for use by the LSC. 



Not 

at 

all 



To a 

great Don’t 
extent know N/A 



1 2 3 4 5 



6 7 



2. The instructional strategies were consistent with 
investigative mathematics/science. 



2 3 4 5 



6 7 



3. The teacher appeared confident in his/her ability to teach 
mathematics/science. 



2 3 4 5 



6 7 



4. The teacher’ s classroom management style/strategies 
enhanced the quality of the lesson. 



2 3 4 5 



5. The pace of the lesson was appropriate for the developmental 
ley els/needs of the students and the purposes of the lesson. 



2 3 4 5 



6. The teacher took into account prior knowledge of students. 1 2 3 4 5 

7. The teacher’s questioning strategies were likely to enhance the 
development of student conceptual understanding/problem solving 
(e.g., emphasized higher order questions, appropriately used 

“wait time,” identified prior conceptions and misconceptions). 1 2 3 4 5 

8. The lesson was modified as needed based on teacher 

questioning or other student assessments. 1 2 3 4 5 



9. 



2 3 4 5 



6 

6 

6 



6 

6 



7 

7 

7 



7 



7 



B. Synthesis Rating 



1 


2 


3 


4 


5 


Implementation of the 
lesson not at all reflective 
of best practice in 
mathematics/ science 
education 








Implementation of the 
lesson extremely reflective 
of best practice in 
mathematics/science 
education 



C. Supporting Evidence for Synthesis Rating 



0 
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III. Mathematics/Science Content 



A. Ratings of Key Indicators 

1. The mathematics/science content was significant and worthwhile. 

2. The mathematics/science content was appropriate for the 
developmental level of the students in this class. 

3. Students were intellectually engaged with important ideas 
relevant to the focus of the lesson. 

4. Teacher-presented information was accurate. 

5. The teacher displayed an understanding of mathematics/science 
concepts (e.g., in his/her dialogue with students). 

6. Mathematics/science was portrayed as a dynamic body of 
knowledge continually enriched by conjecture, investigation 
analysis, and/or proof/justification. 

7. Elements of mathematical/science abstraction (e.g., symbolic 
representations, theory building) were included when it was 
important to do so. 

8. Appropriate connections were made to other areas of mathematics/ 
science, to other disciplines, and/or to real-world contexts. 

9. The degree of closure or resolution of conceptual understanding 
was appropriate for the developmental levels/needs of the 
students and the purposes of the lesson. 



10 . 



Not 

at 

all 



To a 
great 
extent 



1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 



Don’t 

know 



N/A 

7 



7 

7 



B. Synthesis Rating 



1 


2 


3 


4 


5 | 


Mathematics/science 
content of lesson not at all 
reflective of current 
standards for 
mathematics/science 
education 








Mathematics/science 
content of lesson 
extremely reflective of 
current standards for 
mathematics/science 
education 



C. Supporting Evidence for Synthesis Rating 



70 
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IV. Classroom Culture 



A. Ratings of Key Indicators 

1. Active participation of all was encouraged and valued. 

2. There was a climate of respect for students’ ideas, 
questions, and contributions. 

3. Interactions reflected collaborative working relationships 
among students (e.g., students worked together, talked with 
each other about the lesson). 



Not 

at 

all 



To a 

great Don’t 
extent know N/A 



1 2 3 4 5 6 7 

1 2 3 4 5 6 7 



1 2 3 4 5 



6 7 



4. Interactions reflected collaborative working relationships 

between teacher and students. 1 2 3 4 5 6 7 

5. The teacher’s language and behavior clearly demonstrated 
sensitivity to issues of gender, race/ethnicity, special 
needs, Limited English Proficiency, culture, and/or 

socio-economic status. 7 1 2 3 4 5 6 7 



6. Opportunities were taken to recognize and challenge stereotypes 

and biases that became evident during the lesson. 1 2 3 4 5 

7. The climate of the lesson encouraged students to generate 

ideas, questions, conjectures, and/or propositions. 1 2 3 4 5 

8. Intellectual rigor, constructive criticism, and the challenging 

of ideas were valued. 1 2 3 4 5 

9. 1 2 3 4 5 



6 7 

6 7 

6 7 



B. Synthesis Rating 



1 


2 


3 


4 


5 


Classroom culture 
interferes with student 
learning 








Classroom culture 
facilitates the learning of 
all students 



C. Supporting Evidence for Synthesis Rating 



7 Use 1, “Not at all,” when you have considerable evidence of insensitivity or inequitable behavior, 3, when there are no examples either way; and 5, 
‘To a great extent,” when there is considerable evidence of proactive efforts to achieve equity. 



o 
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V. Overall Ratings of the Lesson 

A. Likely Impact of Instruction on Students’ Understanding of Mathematics/Science 

While the impact of a single lesson may well be limited in scope, it is important to judge whether the lesson 
is helping move students in the desired direction. For this series of ratings, consider all available 
information (i.e., your previous ratings of design, implementation, content, and culture, and the pre- and 
post-observation interviews with the teacher) as you assess the likely impact of this lesson. Feel free to 
elaborate on ratings with comments in the space provided. 

Select the response that best describes your overall assessment of the likely effect of this lesson in each of the 
following areas. 









Mixed or 














Negative 




Neutral 




Positive 


Don’t 








effect 




effect 




effect 


know 


N/A 


1. 


Students’ understanding of mathematics/science as a dynamic 
body of knowledge generated and enriched by investigation. 


o 


o 


o 


o 


o 


o 


O 


2. 


Students’ understanding of important mathematics/science 
concepts. 


o 


o 


o 


o 


o 


o 


O 


3. 


Students’ capacity to carry out their own inquiries. 


o 


o 


o 


o 


o 


o 


o 


4. 


Students’ ability to apply or generalize skills and concepts to 
other areas of mathematics/science, other disciplines, and/or 
real-life situations. 


o 


o 


o 


o 


o 


o 


o 


5. 


Students’ self-confidence in doing mathematics/science. 


o 


o 


o 


o 


o 


o 


o 


6. 


Students’ interest in and/or appreciation for the discipline. 


o 


o 


o 


o 


o 


o 


o 



Comments (optional): 
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B. Capsule Description of the Quality of the Lesson 

In this final rating of the lesson, consider all available information about the lesson, its context and 
purpose, and your own judgment of the relative importance of the ratings you have made. Select the 
capsule description that best characterizes the lesson you observed. Keep in mind that this rating is not 
intended to be an average of all the previous ratings, but should encapsulate your overall assessment of the 
quality and likely impact of the lesson. Please provide a brief rationale for your final capsule description 
of the lesson in the space provided. 

O Level 1: Ineffective Instruction 

There is little or no evidence of student thinking or engagement with important ideas of 
mathematics/science. Instruction is unlikely to enhance students’ understanding of the discipline or to 
develop their capacity to successfully “do” mathematics/science. Lesson was characterized by either 
(select one below): 

O Passive “Learning” 

Instruction is pedantic and uninspiring. Students are passive recipients of information from the 
teacher or textbook; material is presented in a way that is inaccessible to many of the students. 

O Activity for Activity’s Sake 

Students are involved in hands-on activities or other individual or group work, but it appears to 
be activity for activity’s sake. Lesson lacks a clear sense of purpose and/or a clear link to 
conceptual development. 

O Level 2: Elements of Effective Instruction 

Instruction contains some elements of effective practice, but there are substantial problems in the 
design, implementation, content, and/or appropriateness for many students in the class. For example, 
the content may lack importance and/or appropriateness; instruction may not successfully address the 
difficulties that many students are experiencing, etc. Overall, the lesson is quite limited in its 
likelihood to enhance students’ understanding of the discipline or to develop their capacity to 
successfully “do” mathematics/science. 

O Level 3: Beginning Stages of Effective Instruction (Select one below.) 

O Low 3 O Solid 3 O High 3 

Instruction is purposeful and characterized by quite a few elements of effective practice. Students are, 
at times, engaged in meaningful work, but there are some weaknesses in the design, implementation, 
or content of instruction. For example, the teacher may short-circuit a planned exploration by telling 
students what they “should have found”; instruction may not adequately address the needs of a 
number of students; or the classroom culture may limit the accessibility or effectiveness of the lesson. 
Overall, the lesson is somewhat limited in its likelihood to enhance students’ understanding of the 
discipline or to develop their capacity to successfully “do” mathematics/science. 

O Level 4: Accomplished, Effective Instruction 

Instruction is purposeful and engaging for most students. Students actively participate in 
meaningful work (e.g., investigations, teacher presentations, discussions with each other or the 
teacher, reading). The lesson is well-designed and the teacher implements it well, but adaptation of 
content or pedagogy in response to student needs and interests is limited. Instruction is quite likely to 
enhance most students’ understanding of the discipline and to develop their capacity to successfully 
“do” mathematics/science. 

O Level 5: Exemplary Instruction 

Instruction is purposeful and all students are highly engaged most or all of the time in meaningful 
work (e.g., investigation, teacher presentations, discussions with each other or the teacher, reading). 
The lesson is well-designed and artfully implemented, with flexibility and responsiveness to students’ 
needs and interests. Instruction is highly likely to enhance most students' understanding of the 
discipline and to develop their capacity to successfully “do” mathematics/science. 

Please provide your rationale for the capsule rating: 
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1997-98 Local Systemic Change 
Post-Classroom Observation Interview 

After you have expressed appreciation to the teacher for allowing you to observe the class, ask 
the following questions: 

1 . Were there any ways in which the lesson was different from what you had planned? 

2. What did this lesson tell you about what your students are learning and still need to learn 
in mathematics/science? 

PROBE: How do you plan to further assess the students’ learning? 

3. What challenges have you faced in encouraging your students to be actively engaged in 
this mathematics/science class? 

PROBE: How have you approached these challenges? 

4. What is the next step for this class? 



O 
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NOTE: 



This form is included for information purposes only. Evaluators will need to 
complete the form on the Web or request scannable forms from HRL 



1998-99 Local Systemic Change 
Professional Development Observation Protocol 1 



Background Information 



Project Date of Observation 

If you are submitting two professional development observations 
for this date , indicate whether this was the first or second 
Location session observed. O 1 st O 2 nd 



Observer Approximate Duration of Observation 2 : 

O 1 hour O 3 hours 

O 2 hours O half day 

Observer's Role in Project: O Project Evaluator O Local Observer 

O Lead 
O Other 

Subject Targeted by session O Mathematics O Science O Both Mathematics and Science 

Section One: Contextual Background And Activities 

In this section, please fill in the circles that best describe the session. For each item, be sure to fill in all 
responses that apply. 

I. Session Demographics 

A. What is the total number of participants attending this session? 

O 1-5 O 6-10 O 11-20 O 21-50 O 51-100 O More than 100 



B. Please describe the targeted subject(s)/grade level(s)/audience for this professional development session. 

1. This session was intended to improve the teaching of: (select all that apply) 

O Elementary science O Elementary mathematics 

O Middle grades science O Middle grades mathematics 

O High school science O High school mathematics 

2. Participants were: 

O Lead teachers for the L SC projects 
O Other (non-lead) teachers 
O Administrators 

O Other (Please specify.) 



Be sure you have read the “1998-99 Local Systemic Change Professional Development Observations: Guidelines for Evaluators” and have completed the 
“Pre-Observation Interview with Professional Development Facilitator” before observing the session. 

2 

The observation recorded on this form should be no less than one hour and no more than half a day. 
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C. Please describe the major presenters/facilitators 3 for this particular one-hour to half-day 
professional development session. 



1 . 



Indicate the number of presenters/facilitators in each gender and race/ethnicity category. 





African-American 
(not Hispanic-origin) 


American Indian 
or Alaskan Native 


Asian or 
Pacific Islander 


Hispanic 


White (not 
Hispanic origin) 


Other 


Male 














Female 















2. Indicate the number of presenters/facilitators for this particular session with each affiliation. 



Regular 

Full-Time 

or 

Part-Time 

Classroom 

Teachers 


Teachers 

on 

Special 

Assignment 4 


District 

Mathematics/ 

Science 

Supervisor 


Other 

District 

Personnel 


University 

Mathematics/ 

Science 

Faculty 


University 

Mathematics/ 

Science 

Education 

Faculty 


Business 

Industry 

Mathematicians/ 

Scientists 


Other 

Non- 

District 

Personnel 



















II. Session Context 

In a few sentences, describe the session you observed. Include: (a) whether the observation covered a partial or 
complete session, (b) whether there were multiple break-out sessions, and (c) where this session fits in the project's 
sequence of professional development for those in attendance. 

HI. Session Focus 

A. Indicate the primary intended purpose(s) of this professional development session based on the information 
provided by the project staff. 

O 1. Increasing mathematics/science content knowledge of teachers and/or teacher leaders. (Be sure to complete 
Category III: Mathematics/ Science Content and Category VII A: Likely Impact on Participants' Capacity 
to Provide High-Quality Mathematics! Science Education, in Section Two of the protocol.) 

O 2. Explicit attention to classroom pedagogy. (Be sure to complete Category IV: Exploring Pedagogy and 
Category VII A: Likely Impact on Participants' Capacity to Provide High-Quality Mathematics/ Science 
Education, in Section Two of the protocol.) 

O a. Creating a vision of effective mathematics/science instruction 
O b. Understanding student thinking/leaming about mathematics/science content 
O c. Learning how to use specific instructional materials in the classroom 
O d. Learning how to use technology in the classroom. 

O e. Learning pedagogical/classroom management strategies 
O f. Considering issues of access, equity, and diversity 
O g. Designing or scoring student assessments 

O h. Considering issues of scope and sequence (e.g., K-12 curricular frameworks) 

O 3. Explicit attention to strategies/issues/roles of teacher leaders. (Be sure to complete Category V: Leadership 
Content and Category VII. B: Likely Impact on Participants' Leadership Capacity, in Section Two of the 
protocol.) 

O 4. Other major purposes: 

O a. Orientation to the project 

O b. Assessing participants’ knowledge/skills 

O c. Building professional networks among educators 

O d. Promoting/exploring reflective practice 

O e. Developing the capacity of participants to use technology 

O f. Involving administrators and/or other school/district personnel in the reform process 



3 In some instances this may not be appropriate, e.g., a session in which a group of teachers meets after school to discuss their action research projects may have no 
presenters or facilitators. In these instances, please leave the presenters/facilitators cells blank. 

4 Defined as teachers released full-time from classroom responsibilities to work on assignments such as the LSC project. 
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B. Indicate the major* mathematics/science content area(s) addressed in this professional development session, whether 
increasing content knowledge was a stated purpose or the mathematics/science content was simply a vehicle for 
achieving other purposes. 

O 1. Numeration and number theory 
O 2. Computation 
O 3. Estimation 
O 4. Measurement 
O 5. Patterns and relationships 
O 6. Pre-algebra 
O 7. Algebra 

O 8. Geometry and spatial sense 
O 9. Functions (including trigonometric 
functions) and pre-calculus 
concepts 

O 10. Data collection and analysis 
O 11. Probability 

O 12. Statistics (e.g., hypothesis tests, 
curve-fitting, and regression) 

O 13. Topics from discrete mathematics 
(e.g., combinatorics, graph theory, 
recursion) 

O 14. Mathematical structures (e.g., 
vector spaces, groups, rings, 
fields) 

O 15. Calculus 

IV. Professional Development Activities 

A. Were any of the instructional materials intended for classroom use as part of the LSC (e.g., FOSS; Insights; STC; 
Investigations in Number, Data, and Space; Connected Math; IMP; SEPUP) a focus of the professional 
development session? 

O No 

O Yes Please specify. 

B. Indicate the major * activities of participants in this session. When choosing an "umbrella" category, be sure to indicate 
subcategories that apply as well. For example, if you mark "formal presentations," indicate by whom. 

O 1. Listened to a formal presentation by: O 2. Engaged in discussions/seminars/reporting out structured as: 

O a. Presenter/facilitator O a. Whole group led by presenter/facilitator 

O b. Participants ) O b. Whole group led by participants) 

O c. Small groups/pairs 

O 3. Engaged in problem solving/investigation 
O 4. Read about disciplinary content, pedagogy, or reform issues 
O 5. Wrote about disciplinary content, pedagogy, or reform issues 

C. Comments 

Please provide any additional information you consider necessary to capture the activities or context of this professional 
development session. Include comments on any feature of the session that is so salient that you need to get it "on the table" 
right away to help explain your ratings. 



O 16. Life Science (Please specify.) 

O 17. Physical science (Please specify.) 

O 18. Earth/space sciences 
O a. Astronomy 
O b. Oceanography 
O c. Geology 
O d. Meteorology 
O e. Environmental science 
O 19. Engineering and design principles 
O 20. History of mathematics/science 

□ Mathematics/science concepts were not included 

as either an explicit focus or a vehicle for achieving other 
professional development purposes 



“Major” means was used or addressed for a substantial portion of the session; if you were describing the session to someone, this feature would help characterize it 
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Section Two: Ratings 



In Section One of this form, you documented what occurred in the session. In this section, you are asked to 
use that information, as well as any other pertinent observations, to rate each of a number of key indicators in 
six different categories, from 1 (not at all) to 5 (to a great extent). 

Note that any one session is not likely to provide evidence for every single indicator; use 6, “Don't know” 
when there is not enough evidence for you to make a judgment. Use 7, “N/A” (Not Applicable) when you 
consider the indicator inappropriate given the purpose and context of the session. For example, a session 
that focuses on engaging teachers in mathematics/science inquiry may choose not to address classroom 
applications. In that case, key indicator #7 for Design, “The design of the session provided opportunities for 
teachers to consider classroom applications of resources, strategies, and techniques,” would be rated “N/A,” 
rather than “not at all.” 

Similarly, there may be entire rating categories that are not applicable to a particular session. For example, 
categories HI, IV, and V (Content) and Overall Ratings VILA (Impact on Participants’ Capacity to Provide 
High Quality Mathematics/Science Education) and VEIB (Impact on Participants’ Leadership Capacity) each 
have a box to check when the entire rating category is judged to be inappropriate for the session 6 . Categories 
I (Design), II (Implementation), and VI (Culture of the Professional Development Session) are ones in which 
specific indicators may be “not applicable,” but the overall category should routinely be rated for any 
observation. 

Note that you may list any additional indicators you consider important in capturing the essence of this 
session and rate these as well. 

Use your “Ratings of Key Indicators” (Part A) to inform your “Synthesis Ratings” (Part B). It is important 
to indicate in “Supporting Evidence for Synthesis Ratings” (Part C) what factors were most influential in 
determining your synthesis ratings and to give specific examples or quotes to illustrate those factors. Section 
Two concludes with ratings of the likely impact of professional development, and a capsule description of the 
session. 



6 



In most cases, the categories you rate will be consistent with the purposes marked in Section One. Part HI. A. 1 through 3. 
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Not 

at 

all 



Design 

A. Ratings of Key Indicators 

1 . The design of the session incorporated tasks, roles, and 
interactions consistent with a spirit of investigation. 

2. The instructional strategies and activities used in this 
session reflected attention to participants’ experience, 
preparedness, and/or learning styles. 

3. The session effectively built on participants’ knowledge of 
content, teaching, learning, and/or the reform process. 

4. The strategies in this session were appropriate for accomplishing 
the purposes of the LSC professional development. 

5. The design of the session reflected careful planning and 
organization. 

6. The design of the session encouraged a collaborative 
approach to learning. 

7. The design of the session provided opportunities for teachers 
to consider classroom applications of resources, strategies, 
and techniques. 

8. Adequate time and structure were provided for reflection. 

9. Adequate time and structure were provided for participants 
to share experiences and insights. 



10. Adequate time and structure were provided for wrap-up and closure. 1 



11 . 



To a 
great 
extent 



3 4 5 



4 5 
4 5 
4 5 
4 5 
4 5 

4 5 
4 5 

4 5 
4 5 
4 5 



Don’t 

know 



N/A 



6 

6 

6 

6 



B. Synthesis Rating 



1 


2 


3 


4 


5 


Design of the session not 
at all reflective of best 
practice for professional 
development. 








Design of the session 
extremely reflective of 
best practice for 
professional development. 



C. Supporting Evidence for Synthesis Rating 
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II. Implementation 



A. Ratings of Key Indicators 

1. The session effectively incorporated instructional strategies 
that were appropriate for the purposes of the professional 
development session. 



Not 

at 

all 



To a 

great Don’t 
extent know N/A 



1 2 3 4 5 



6 7 



2. The session effectively modeled questioning strategies that are 
likely to enhance the development of conceptual understanding 
(e.g., emphasis on higher-order questions, appropriate use of 
“wait time,” identifying prior conceptions and misconceptions.) 



2 3 4 5 



6 7 



3. The pace of the session was appropriate for the purposes of 
the professional development and the needs of adult learners. 



2 3 4 5 



6 7 



4. The session modeled effective assessment strategies. 



2 3 4 5 



6 7 



5. The facilitator(s)’ background, experience, and/or expertise 
enhanced the quality of the session. 



2 3 4 5 



6 7 



6. The facilitator(s)’ management style/strategies enhanced the 
quality of the session. 



2 3 4 5 



7. 



2 3 4 5 



6 . 7 



B. Synthesis Rating 



1 


2 


3 


4 


5 


Implementation of the 
session not at all reflective 
of best practice for 
professional development; 








Implementation of the j 

session extremely 
reflective of best practice 
for professional 
development 



C. Supporting Evidence for Synthesis Rating 
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HI. Mathematics/Science Content 



Complete this category if: a) increasing mathematics/science content knowledge was a key purpose of the session; b) 
mathematics/science content was a vehicle for accomplishing other professional development purposes; or c) inadequate 
coverage in this area acted as a barrier to accomplishing other stated purposes of the session. If none of these apply, 
check here □ and skip to category IV. 



A. Ratings of Key Indicators 

1 . Mathematics/science content was appropriate for the purposes 
of the professional development session and the backgrounds 
of the participants. 

2. Mathematics/science content was sound and appropriately 
presented/explored. 

3. Participants were intellectually engaged with important 
ideas relevant to the focus of the session. 

4. Facilitators) displayed an understanding of mathematics/science 
concepts (e.g., in their dialogue with participants). 

5. Mathematics/science was portrayed as a dynamic body of 
knowledge continually enriched by conjecture, investigation, 
analysis, and/or proof/justification. 

6. Depth and breadth of attention to mathematics/science content was 
appropriate for the purposes of the session and participants’ needs. 

7. Elements of mathematical/scientific abstraction (e.g., symbolic 
representations, theory building) were included when it was 
important to do so. 

8. Appropriate connections were made to other areas of mathematics/ 
science, to other disciplines, and/or to real-world contexts. 

9. Degree of closure or resolution of mathematics/science 
conceptual understanding was appropriate for the purposes of the 
session and the needs of adult learners. 



10 . 



Not 

at 

all 



To a 
great 
extent 



4 5 



4 5 



4 5 



4 5 



4 5 



4 5 



1 2 3 4 5 



4 5 



4 

4 



Don’t 

know 



N/A 



B. Synthesis Rating 



1 


2 


3 


4 


5 


Mathematics/science 
content of session not at 
all reflective of current 
standards for 
mathematics/science 
education 








Mathematics/science 
content of session 
extremely reflective of 
current standards for 
mathematic s/science 
education 



C. Supporting Evidence for Synthesis Rating 
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IV. Exploring Pedagogy 



Complete this category if: a) exploring classroom practice was a key purpose of the session; or b) lack of/inadequate 
coverage in this area acted as a barrier to accomplishing other stated purposes of the session. If neither of these apply, 
check here □ and skip to category V. 



A. Ratings of Key Indicators 

1. Depth and breadth of attention to student thinking/leaming were 
appropriate for the purposes of the session and participants’ needs. 



Not 

at 

all 



To a 

great Don’t 
extent know N/A 



1 2 3 4 5 



6 7 



2. Depth and breadth of attention to classroom 

strategies were appropriate for the purposes of the session and 
participants’ needs. 



2 3 4 5 



6 7 



3. Depth and breadth of attention to instructional materials intended for 
classroom use were appropriate for the purposes of the session and 

participants’ needs. 1 2 3 4 5 



6 7 



4. Facilitators) displayed an understanding of pedagogical 
concepts (e.g., in their dialogue with participants). 

5. Participants were intellectually engaged with important 
ideas relevant to classroom practice. 

6. Degree of closure or resolution of conceptual understanding 
about classroom practice was appropriate for the purposes 
of the session and the needs of adult learners. 



1 2 3 4 5 



6 7 



1 2 3 4 5 



6 7 



1 2 3 4 5 



6 7 



7. 



2 3 4 5 



B. Synthesis Rating 



1 


2 


3 


4 


5 


Pedagogical content of 
session not at all reflective 
of current standards for 
mathematics/scien ce 
education 








Pedagogical content of 
session extremely 
reflective of current 
standards for 
mathematics/ science 
education 



C. Supporting Evidence for Synthesis Rating 
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V. Leadership Content 



Complete this category only if exploring strategies/issues/roles of teacher leaders was a key purpose of the session. If not, 
check here □ and skip to category VI. 



A. 


Ratings of Key Indicators 


Not 






To a 










at 






great 


Don’t 




1. 


Information on principles of effective staff development was 


all 






extent 


know 


N/A 




sound and appropriately presented/explored. 


1 


2 


3 


4 


5 


6 


7 


2. 


Information on strategies for mentoring/coaching peers was 


















sound and appropriately presented/explored. 


1 


2 


3 


4 


5 


6 


7 


3. 


Information on how to be a reform advocate at school/district 


















level was sound and appropriately presented/explored. 


1 


2 


3 


4 


5 


6 


7 


4. 


Facilitator(s) displayed an understanding of leadership concepts 


















(e.g., in their dialogue with participants). 


1 


2 


3 


4 


5 


6 


7 


5. 


Participants were intellectually engaged with important 


















ideas relevant to the focus of the session. 


1 


2 


3 


4 


5 


6 


7 


6. 


Participants were given adequate and appropriate opportunity to 


















consider how the content of the session applies to their particular 


















leadership roles. 


1 


2 


3 


4 


5 


6 


7 


7. 




1 


2 


3 


4 


5 







B. Synthesis Rating 



1 


2 


3 


4 


5 


Leadership content not at 
all appropriate for 
preparing participants to 
be school/district leaders 
of mathematics/science 
education 








Leadership content highly 
appropriate for preparing 
participants to be 
school/district leaders of 
mathematics/science 
education 



C. Supporting Evidence for Synthesis Rating 
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VI. Culture of the Professional Development Session 



Al. Ratings of Key Indicators 

1 . Active participation of all was encouraged and valued. 

2. There was a climate of respect for participants’ experiences, 
ideas, and contributions. 

3. Interactions reflected collaborative working relationships 
among participants. 

4. Interactions reflected collaborative working relationships 
between facilitators ) and participants. 

5. Participants were encouraged to generate ideas, questions, 
conjectures, and propositions. 

6. Investigation and risk-taking were valued. 



Not 






To a 






at 






great 


Don’t 




all 






extent 


know 


N/A 


1 


2 


3 


4 5 


6 


7 


1 


2 


3 


4 5 


6 


7 


1 


2 


3 


4 5 


6 


7 


1 


2 


3 


4 5 


6 


7 


1 


2 


3 


4 5 


6 


7 


1 


2 


3 


4 5 


6 


7 



A2. Respect for Diversity 

An atmosphere of respect and appreciation for diversity (gender, race/ethnicity, and/or cultural background) can greatly 
enhance the culture of a professional development session. In contrast, fostering stereotypes or exhibiting other overt 
insensitivity are likely to have the opposite effect. If the culture of this session was affected (positively or negatively) by 
such instances, please check here □ and explain: 



B. Synthesis Rating 



1 


2 


3 


4 


5 


Culture of the session 
interferes with 
engagement of 
participants as members of 
a professional learning 
community 








Culture of the session 
facilitates engagement of 
participants as members of 
a professional learning 
community 



C. Supporting Evidence for Synthesis Rating 
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VII. Overall Ratings of the Session 



While the impact of a single professional development session may well be limited in scope, it is important to judge whether 
the session is helping move participants in the desired direction. For ratings in Sections A and B below, consider all available 
information (i.e., your previous ratings of design, implementation, content, and culture; related interviews; and your 
knowledge of the overall professional development program) as you assess the likely impact of this session. Feel free to 
elaborate on ratings with comments in the space provided. 

A. Likely Impact on Participants’ Capacity to Provide High Quality Mathematics/Science 
Education 

Consider the likely impact of this session on the participants’ capacity to provide high quality mathematics/science 
education. Select the response that best describes your overall assessment of the likely effect of this session in each of the 
following areas. 

□ Not applicable (The session did not focus on building capacity for classroom instruction.) 



Mixed or 







Negative 




Neutral 




Positive 


Don’t 








effect 




effect 




effect 


know 


N/A 


1 . 


Participants’ ability to identify and understand important 


















ideas of mathematics/science. 


o 


o 


o 


o 


o 


o 


o 


2. 


Participants’ understanding of mathematics/science as a dynamic 


















body of knowledge generated and enriched by investigation. 


o 


o 


o 


o 


o 


o 


o 


3. 


Participants’ understanding of how students learn. 


o 


o 


o 


o 


o 


o 


o 


4. 


Participants’ ability to plan/provide high quality mathematics/ 


















science classroom instruction. 


o 


o 


o 


o 


o 


o 


o 


5. 


Participants’ ability to implement the designated 


















instructional materials. 


o 


o 


o 


o 


o 


o 


o 


6. 


Participants’ self-confidence as mathematics/science instructors. 


o 


o 


o 


o 


o 


o 


o 


7. 


Professional networking among participants with regard to 


















mathematics/science instruction. 


o 


o 


o 


o 


o 


o 


o 



Comments (optional): 
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B. Likely Impact on Participants 5 Leadership Capacity 



1 . 

2 . 

3 . 

4 . 

5 . 

6 . 

7 . 

8 . 

9 . 

10 . 



If the session included teacher leaders, consider the likely impact of this session on their leadership capacity. Select the 
response that best describes your overall assessment of the likely effect of this session in each of the following areas. 
Please note that even if an element was not addressed explicitly, it might have a negative or positive effect on leadership 
development, depending on whether it was modeled well or poorly. 



□ Not applicable (The session did not include teacher leaders.) 



Mixed or 





Negative 




Neutral 




Positive 


Don’t 






effect 




effect 




effect 


know 


N/A 


Leaders’ knowledge and understanding of 
mathematics/science. 


o 


o 


o 


o 


o 


o 


o 


Leaders’ knowledge and understanding of effective classroom 
practice. 


o 


o 


o 


o 


o 


o 


o 


Leaders’ ability to convey to others a vision of effective 
mathematics/science classrooms. 


o 


o 


o 


o 


o 


o 


o 


Leaders’ understanding of teachers’ prior knowledge 
and areas where teachers have difficulty. 


o 


o 


o 


o 


o 


o 


o 


Leaders’ understanding of adult learners. 


o 


o 


o 


o 


o 


o 


o 


Leaders’ understanding of the reform process. 


o 


o 


o 


o 


o 


o 


o 


Leaders’ understanding of important strategies for reform 
of mathematics/science education. 


o 


o 


o 


o 


o 


o 


o 


Leaders’ ability to plan/implement exemplary 
professional development. 


o 


o 


o 


o 


o 


o 


o 


Participants’ confidence in serving as effective leaders. 


o 


o 


o 


o 


o 


o 


o 


Professional networking among participants with regard to 
leadership. 


o 


o 


o 


o 


o 


o 


o 



Comments (optional): 
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C. Capsule Description of the Quality of the Professional Development Session 

In this final rating of the session, consider all available information about the session, its context and purpose, and your own 
judgment of the relative importance of the ratings you have made. Select the capsule description that best characterizes the 
session you observed. Keep in mind that this rating is not intended to be an average of all the previous ratings, but should 
encapsulate your overall assessment of the quality and likely impact of the session. Please provide a brief rationale for your final 
capsule description of the session in the space provided. 



O Level 1: Ineffective Professional Development 

There is little or no evidence of participant thinking or engagement with important ideas of mathematics/science education 
Session is unlikely to enhance the capacity of participants to provide high quality mathematics/science education or to be 

effective leaders of mathematics/science education in the district(s). Professional development appears to be either f select 
one below): v 

O Passive “Learning” 

Session is pedantic and uninspiring. Participants are passive recipients of information; material is presented in a way 
that is inaccessible to or inappropriate for many of the participants. 

O Activity for Activity’s Sake 

Participants are involved in hands-on activities or other individual or group work, but it appears to be activity for 
activity s sake. Session lacks a clear sense of purpose and/or a clear link to the conceptual development of participants. 

O Level 2: Elements of Effective Professional Development 

Session contains some elements of effective practice in professional development, but there are substantial problems in the 
design, content, and/or implementation given the purposes of the session. For example, the content is presented in a way 
at would reinforce misconceptions or the pace is clearly too rapid for meaningful participant engagement. Overall, the 
session is quite limited in its likelihood to enhance the capacity of most participants to provide high quality 
mathematics/science education or to be effective leaders of mathematics/science education in the district(s). 

O Level 3: Beginning Stages of Effective Professional Development (Select one below.) 

O Low 3 O Solid 3 O High 3 

Professional development is purposeful and at times effective, but there are some weaknesses in the design, content, or 
implementation of die session. For example, participants’ expertise is not well-utilized; or participants are not given 
sufficient opportunity to reflect on what they are learning. Overall, the session is somewhat limited in its likelihood to 
enhance the capacity of participants to provide high quality mathematics/science education or to be effective leaders of 
mathematics/science education in the district(s). 



O Level 4: Accomplished, Effective Professional Development 

Facilitation is skillful and participants are engaged in purposeful work (e.g., investigations, discussions, presentations, 
reading) designed to enhance their understanding of important mathematics/science concepts, processes, or pedagogy or to 
enhance dieir leadership skills. The facilitator(s) implement the professional development session well and participants’ 
contributions are valued, but adaptation of content or format in response to participants’ needs and interests may be limited 
1 he session is quite likely to enhance the capacity of most participants to provide high quality mathematics/science 
education or to be effective leaders of mathematics/science education in the district(s). 

O Level 5: Exemplary Professional Development 

Facilitation is skillful, and participants are highly engaged in purposeful work (e.g., investigations, discussions, 
presentations, reading) designed to enhance their understanding of important mathematics/science concepts, processes, or 
pedagogy, or to enhance their leadership skills. The session is artfully implemented, with flexibility and responsiveness to 
participant needs/mterests. The session is highly likely to enhance the capacity of participants to provide high quality 
mathematics/science education or to be effective leaders of mathematics/science education in the district(s). 

Please provide your rationale for the capsule rating: 
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